Adventures using Availability Groups and RBS with SharePoint 2013

The concept behind a remote blob storage is pretty simple, see this for instance. I just want to talk about the myriad issues we've had when using RBS. with availability groups.

Our database setup uses Availability Groups, which, and this is controversial, is a cheap cluster. I do get that there are advantages to availability groups but these seem to be outweighed by the disadvantages. I know this is just my opinion and that I also know nothing about availability groups, HA clusters or anything in general, thank you for pointing out.

So what are the drawbacks of AGs?

• Official Support is patchy. e.g. in Dynamics CRM 2013 one is forced to update the database directly.
• Performance can be an issue as the database is always using at least two boxes.
• Stuff flat out refuses to work, e.g. RBS Maintainer, various SharePoint database related operations.

To the AG mix we introduced RBS and this is were things started to go horribly wrong for us.

The first issue we encountered was the inability to delete a content database from SharePoint, which is not a major issue but it's really annoying.

The second issue was that the RBS maintianer would not work, so the storage requirements would just keep growing. This might not be an issue if you don't plan to archive your documents, but our DB had ~500GB of docs, about 2/3 of which were old but for contractual reasons needed to be kept.

This effectively put a nail in the coffin of the RBS + AG combo but there is more.

In order to load the ~500 GB document, we had a tool to upload the documents to SharePoint. This tool was multi-threaded and it essentially read the documents from the source DB and uploaded them to SharePoint, using the SharePoint CSOM model.

At this point, it's worth mentioning that our hosting provider does not guarantee any sort of performance level, too long to explain.

A couple of weeks back, with RBS on the database, we did a trial run of the upload and we were hitting very poor rates, ~ 4 GB per hour.

Last week, after RBS had been disabled and the content databases recreated, we tried a second trial run and the speed jumped to ~ 20 GB per hour.

I can't say that our RBS configuration was perfect, I think the threshold was on the low side (128 KB) but even so, the speed increase has been massive.

It actually gets better, because the 4 GB per hour figure was using both servers in the farm, whereas the 20 GB per hour figure was simply using one.

yes, yes, I know our hosting provider is crap and 128 KB is below the recommendation, but a 5 fold increase in transfer rates and a lowering of the error rate to almost zero is something that should be considered.

Integrating MS SharePoint with MS Dynamics CRM 2011/2013 - User Permissions

One of the things that seems to come up time and again on any integration of MS Dynamics CRM and SharePoint is the issue of user permissions.

Generally speaking it would be nice to be able to control access to SharePoint based upon the permissions the user has in MS Dynamics CRM. Alas, this is not possible without writing a bit code. (I've not investigated the Server to Server Integration yet as it seems to be available for online Dynamics CRM only)

The way I have done this, is by using an HttpModule deployed to the Ms SharePoint servers to check whether the user making the request to the MS SharePoint site has actually got access to the record in MS Dynamics CRM itself.

In our case this is pretty straight forward as we only store documents for a single entity, but there is nothing in principle to rule out an expansion to multiple entities.

Depending on the usage, caching will need to be a serious consideration, as performance could be impacted, but I have not thought about it too much yet.

The following assumptions have been made about the integration between MS Dynamics CRM and MS SharePoint:

1. A document library exists and is named with the entity schema.
2. Each entity record in MS Dynamics CRM has a single folder in MS SharePoint and this folder is named with the GUID of the record. 
3. Entity Records in MS Dynamics CRM are not shared.

This is the code for the module itself:

using log4net;
using Microsoft.Crm.Sdk.Messages;
using Microsoft.Xrm.Sdk.Client;
using Microsoft.Xrm.Sdk.Query;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Net;
using System.Security.Principal;
using System.ServiceModel.Description;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Web;

namespace CRMUserPermissions
{
    public class CRMUserPermissions : IHttpModule
    {

        public static ConcurrentDictionary<string, Guid> userIds = new ConcurrentDictionary<string, Guid>();

        const string GuidPattern = @"(/|%2f)([A-F0-9]{8}(?:-[A-F0-9]{4}){3}-[A-F0-9]{12})";

        const string UserIdQuery = @"<fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
  <entity name='systemuser'>   
    <attribute name='systemuserid' />
    <attribute name='domainname' />
    <filter type='and'>
      <condition attribute='domainname' operator='eq' value='{0}' />
    </filter> 
  </entity>
</fetch>";

        public void Dispose()
        {
        }

        public void Init(HttpApplication context)
        {
            context.PostAuthenticateRequest += new EventHandler(context_PostAuthenticateRequest);
        }

        void context_PostAuthenticateRequest(object sender, EventArgs e)
        {
            HttpApplication app = sender as HttpApplication;
            HttpContext context = app.Context;

            if (IsRequestRelevant(context))
            {
                try
                {

                    string user = HttpContext.Current.User.Identity.Name.Split('|').Last();

                    var service = CrmService.GetService();

                    string url = app.Request.Url.ToString();

                    if (!userIds.ContainsKey(user))
                    {
                        string query = string.Format(UserIdQuery, user);
                        var userId = service.RetrieveMultiple(new FetchExpression(query)).Entities.SingleOrDefault();
                        userIds.TryAdd(user, userId.Id);
                    }

                    var record = GetRecordInfo(url);

                    RetrievePrincipalAccessRequest princip = new RetrievePrincipalAccessRequest();
                    princip.Principal = new EntityReference("systemuser", userIds[user]);

                    princip.Target = new EntityReference(record.Item1, record.Item2);

                    var res = (RetrievePrincipalAccessResponse)service.Execute(princip);

                    if (res.AccessRights == AccessRights.None)
                    {
                        app.Response.StatusCode = 403;
                        app.Response.SubStatusCode = 1;
                        app.CompleteRequest();
                    }
   
                }
                catch (Exception)
                {
                    app.Response.StatusCode = 403;
                    app.Response.SubStatusCode = 1;
                    app.CompleteRequest();
                }
            }
        }
    }
}

A few comments are in order since I'm not including all methods.

IsRequestRelevant(context): This method checks that the user is authenticated and that the request is for documents relating to an entity we want to control access via this method.

CrmService.GetService(); This method just returns an OrganizationServiceProxy.
GetRecordInfo(url); This method works out the record guid and what type of entity it is.

It would probably be a better idea to get all, or some, users and cache them on receipt of the first query.

Depending on the system's usage profile different types of caching make more or less sense. For instance, if users tend to access multiple documents within a record in a relatively short time, say 10 minutes, then it makes sense to cache the records and the user's right to them but if users only tend to access a single document within a record, this would make less sense. Consideration needs to be given to memory pressures that caching will create if not using a separate caching layer, such as Redis.

The simplest way of deploying this httpModule is by installing the assembly in the GAC and then manually modifying the web.config of the relevant MS SharePoint site by adding the httpModule to the modules in configuration/system.webServer/modules:

<add name="CRMUserPermissions" type="CRMUserPermissions.CRMUserPermissions, CRMUserPermissions,,Version=1.0.0.0, Culture=neutral, PublicKeyToken=87b3480442bff091"></add>

I will post how to do this properly, i.e. by packaging it up in a MS SharePoint solution in an upcoming post.

MS Dynamics CRM - Blocking Direct Access to SharePoint

In this project I'm working on we've had a requirement to block direct access to SharePoint, in other words. Our users can only access their documents through CRM.

There are various ways, in which this can be achieved and today I will be discussing achieving this by leveraging ARR.

We have a similar architecture to this:

We have a Sharepoint site collection and every document is stored in this site collection, so our users would go https://sp.dev.local/sites/spsite/<crm_entity>/ to access documents for <crm_entity>

This is actually a somewhat irritating requirement because IE behaves differently than Firefox and Chrome. We're lucky enough not to have to support Opera and Safari as well, nothing wrong with these browsers, but 5 browsers would drive our testers crazy.

In any case, we've configured two farms, CRM and SHAREPOINT, so we need rules for those.

So for CRM we have:

<rule name="CRM Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="false">
    <match url="*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
        <add input="{HTTP_HOST}" pattern="*crm.dev.local" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://CRM/{R:0}" />
</rule>

And for Sharepoint we have:

<rule name="Sharepoint Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
        <add input="{HTTP_HOST}" pattern="*sp.dev.local" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
</rule>

So far so good, this is where things start to get interesting. We want to block direct access to SP for IE browsers, which we can achieve like this:

 
<rule name="IE - Allow Access to SharePoint grid in CRM" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*sites/SPSITE/crmgrid*" />
    <conditions logicalGrouping="MatchAny" trackAllCaptures="false">
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
</rule>    

and

<rule name="IE - Block Direct Access to SharePoint" stopProcessing="true">
    <match url="sites\/SPSITE" />
    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
        <add input="{HTTP_REFERER}" pattern="^/?$" negate="true" />
        <add input="{HTTP_USER_AGENT}" pattern="MSIE" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="CustomResponse" statusCode="403" subStatusCode="1" statusReason="IE" statusDescription="Direct Access to SHAREPOINT is not permitted" />
</rule>

The first rules allows traffic if it's trying to access SP through the List component, this allows the SharePoint CRM List component to work.

The second rule will block access for IE user agents ( i.e containing MSIE) and where the referer is not empty. It will stop processing if there is a match.

For some reason, I.E. blanks the referer when accessing documents in SharePoint from the list component but crucially fills it in if accessing documents directly from SharePoint.

Firefox and Chrome will have the correct referer, i.e. sp.dev.local/sites/spsite/crmgrid, so there is a single rule:

<rule name="Other Browsers - Block Direct Access to SharePoint" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
    <match url="*sites/SPSITE*" />
    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
        <add input="{HTTP_REFERER}" pattern="*dev.local/sites/SPSITE/crmgrid*" negate="true" />
        <add input="{HTTP_USER_AGENT}" pattern="*MSIE*" negate="true" />
    </conditions>
    <serverVariables>
    </serverVariables>
    <action type="CustomResponse" statusCode="403" subStatusCode="2" statusReason="Other" statusDescription="Direct Access to SHAREPOINT is not permitted" />
</rule>

Just need to make sure that the user agent is not from IE. The rule will stop processing if there is a match.

Rules.xml can be found below, with the rules in the correct order.

Thus effectively we do the following:

CRM -> Rewrite to  CRM Farm
SP - crm grid?  -> Rewrite to SP Farm
SP - IE and empty Referer -> Rewrite to SP Farm
SP - Other Browser and correct Referer -> Rewrite to SP Farm
SP -> Rewrite to Farm

The last rule is only needed if there are other sites in SP that the users might need to access.

<?xml version="1.0" encoding="UTF-8"?>
<appcmd>
    <CONFIG CONFIG.SECTION="system.webServer/rewrite/globalRules" path="MACHINE/WEBROOT/APPHOST" overrideMode="Inherit" locked="false">
        <system.webServer-rewrite-globalRules>
            <rule name="CRM Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="false">
                <match url="*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
                    <add input="{HTTP_HOST}" pattern="*crm.dev.local" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://CRM/{R:0}" />
            </rule>
            <rule name="IE - Allow Access to SharePoint grid in CRM" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*sites/SPSITE/crmgrid*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="false">
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
            </rule>
            <rule name="IE - Block Direct Access to SharePoint" stopProcessing="true">
                <match url="sites\/SPSITE" />
                <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                    <add input="{HTTP_REFERER}" pattern="^/?$" negate="true" />
                    <add input="{HTTP_USER_AGENT}" pattern="MSIE" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="CustomResponse" statusCode="403" subStatusCode="1" statusReason="IE" statusDescription="Direct Access to SHAREPOINT is not permitted" />
            </rule>
            <rule name="Other Browsers - Block Direct Access to SharePoint" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*sites/SPSITE*" />
                <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                    <add input="{HTTP_REFERER}" pattern="*dev.local/sites/SPSITE/crmgrid*" negate="true" />
                    <add input="{HTTP_USER_AGENT}" pattern="*MSIE*" negate="true" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="CustomResponse" statusCode="403" subStatusCode="2" statusReason="Other" statusDescription="Direct Access to SHAREPOINT is not permitted" />
            </rule>
            <rule name="Sharepoint Farm Forwarding Rule" enabled="true" patternSyntax="Wildcard" stopProcessing="true">
                <match url="*" />
                <conditions logicalGrouping="MatchAny" trackAllCaptures="true">
                    <add input="{HTTP_HOST}" pattern="*sp.dev.local" />
                </conditions>
                <serverVariables>
                </serverVariables>
                <action type="Rewrite" url="https://SHAREPOINT/{R:0}" />
            </rule>
        </system.webServer-rewrite-globalRules>
    </CONFIG>
</appcmd>

It can be exported with this command:

appcmd.exe list config -section:system.webServer/rewrite/globalRules -xml > rules.xml 

The imported with this command:

appcmd.exe set config /in < rules.xml

Removing HTTP Headers for an ARR/MS Dynamics CRM/Sharepoint 2013 system

We had a pen test carried out last week and one of the outcomes was that we were leaking information with our HTTP headers and they must be removed.

Our environment consists of a web layer (IIS using ARR) and then an app layer (MS Dynamics CRM and MS SharePoint)

Personally I think this is a bit of security through obscurity but needs must so here we go:

X-AspNet-Version Header:

MS Dynamics CRM 2013

In the web.config, <drive>:Program Files\Microsoft Dynamics CRM\CRMWeb  add enableVersionHeader="false" to the httpRuntime element, normally you'll end up with something like this:

<httpRuntime executionTimeout="300" maxRequestLength="32768" requestValidationMode="3.0" encoderType="Microsoft.Crm.CrmHttpEncoder, Microsoft.Crm" enableVersionHeader="false"/>

MS SharePoint 2013

In the web.config,  <drive>:inetpub\wwwroot\wss\VirtualDirectories\80\ add enableVersionHeader="false" to the httpRuntime element, normally you'll end up with something like this:

<httpRuntime maxRequestLength="51200" requestValidationMode="2.0" enableVersionHeader="False" />

X-Powered-By Header:

From IIS Manager -> Server -> HTTP Response Headers

Server Header:

The simplest way I found is to use URL Rewrite to blank this header, which works very well for our system as we're using ARR already so just need to do this one on the web layer ... from IIS Manager -> Url Rewrite -> Add Rule

Select Blank Outbound Rule

Fill in the details as below

Don't forget to click Apply when you've finished.

It's worth pointing out  that this will simply blank out the value of the Server Header, rather than remove it completely.

If you want to remove it completely you will need to install urlscan.

This approach can be used for all the other headers above I suppose.

X-Powered-By: ARR/2.5 Header

From a powershell console with elevated permissions go to C:\Windows\system32\inetsrv and run this command:

.\appcmd.exe set config -section:webFarms /"[name='serverfarmname'].applicationRequestRouting.protocol.arrResponseHeader:false" /commit:apphost

SPApplicationAuthenticationModule: There is no Authorization header, can't try to perform application authentication.

This week we had an interesting issue with one of our SharePoint servers.

We have a two server farm, both servers are full servers that had been installed a couple of months ago and as far as I was aware both servers had been tested, so I was little bit surprised when the farm was tested in anger and we were getting a roughly ~20% failure rate in a process that uploads a document to SharePoint.

After a bit of digging we found that it was due to one of the SharePoint servers. 

We could not even log in to any of sites hosted on the farm if we hit this server. We simply would get a 401 unauthorized error. 

I know we also seem to have a load balancing issue but that's for another day.

Perhaps, unsurprisingly the logs did not show much, so I bumped them up to verbose and here's what we found:

Claims Authentication        SPIisWebServiceAuthorizationManager: Using identity '0#.w|dev\svc-spadm' as the actor identity.
Topology                     WcfReceiveRequest: LocalAddress: 'http://sp02.dev.local:32843/934e0061c6a94255b9ab9e6f2ba45325/SearchService.svc' Channel: 'System.ServiceModel.Channels.ServiceChannel' Action: 'http://tempuri.org/ISearchHealthMonitoringServiceApplication/GetQueryHealthMonitoringSettingsForComponents' MessageId: 'urn:uuid:f00ca305-b1d5-4454-85fa-5f83e7094518'
Monitoring                   Leaving Monitored Scope (ExecuteWcfServerOperation). Execution Time=154.973746663333
Monitoring                   Entering monitored scope (Request (GET:https://<site>)). Parent No
Logging Correlation Data     Name=Request (GET:https://<site>)
Claims Authentication        SPTokenCache.ReadTokenXml: Successfully read token XML ''.
Application Authentication   SPApplicationAuthenticationModule: There is no Authorization header, can't try to perform application authentication.
Authentication Authorization Non-OAuth request. IsAuthenticated=False, UserIdentityName=, ClaimsCount=0
Claims Authentication        Claims Windows Sign-In: Sending 401 for request 'https://<site>' because the user is not authenticated and resource requires authentication.
Monitoring                   Leaving Monitored Scope (Request (GET:https://<site>)). Execution Time=3.75103539695688
Claims Authentication        SPFederationAuthenticationModule.OnEndRequest: User was being redirected to authenticate.
Claims Authentication        Claims Windows Sign-In: Sending 401 for request 'https://<site>' because the user is not authenticated and resource requires authentication.

Clearly, It's not able to authenticate but why? I thought that the lack of authorization header was the clue but nothing I found in Google helped me and then I sort of had a flash of inspiration and decided to check whether the site had Windows Authentication enabled.

Bingo!!!!!  Windows Authentication is Disabled, no wonder nobody could log in :)

After I enabled it and restarted IIS, the second server started working :)

I didn't install SharePoint on these servers and I don't really have that much experience with SharePoint so I'm entirely sure who to blame here, our guys or Microsoft, but it seems to me that since one of the big things with Microsoft is integration with AD, it's just a bit daft that it doesn't turn Windows Authentication on for the SharePoint site by default. 

Maybe it does and it's something that we did.

At any rate, hope this helps.

Brain Dump 7 - Remove User from group in SharePoint

Clue is in the title

In essence below is a method that will remove a user from a group in SharePoint.

User can be of the form domain\user or user@domain

public bool RemoveUserFromSharePointGroup(string userName, string groupName)
{
 var principal = Microsoft.SharePoint.Client.Utilities.Utility.ResolvePrincipal(context, context.Web, userName,
  Microsoft.SharePoint.Client.Utilities.PrincipalType.User, Microsoft.SharePoint.Client.Utilities.PrincipalSource.All,
  context.Web.SiteUsers, false);
  
 context.ExecuteQuery();
 
 if (principal.Value != null)
 {
  string login = principal.Value.LoginName;
  GroupCollection siteGroups = context.Web.SiteGroups;
  Group group = siteGroups.GetByName(groupName);
 
  var query = context.LoadQuery(group.Users.Where(usr => usr.LoginName == login).Include(u => u.LoginName));
 
  context.ExecuteQuery();
 
  User user = query.SingleOrDefault();
 
  if (user != null)
  {
   group.Users.RemoveByLoginName(user.LoginName);
  }
 
  context.ExecuteQuery();
 
 }
}

Validating SharePoint file names in JavaScript

SharePoint limits the valid characters of file names, which can be a problem, so to prevent this issue, we use this function to validated filenames in our Web App that integrates with SharePoint.

var validateFileName = function (value){

var specialCharacters = new RegExp("[\\\\\/:*?\"<>|#{}%~&]");

if (specialCharacters.test(value)) {      
 return true;
}
else{
 return false;
}

}

Only thing to note is that \\\\ is needed to represent \ in the regular expression, see this for more details

Remove Users from group in SharePoint 2013 from Client Object model.

This is the method that we use to remove users from groups in Sharepoint 2013

It's a bit more convoluted than it needs to be as for some reason Sharepoint 2013 appends the claim provider to the username, which means that rather than having:

 dev\duser1  

we have something like:

i:0#.w|dev\duser1  

so in order to ensure that the it will always find the relevant user, we do the extra call to ResolvePrincipal.

using Microsoft.SharePoint.Client;
using Microsoft.SharePoint.Client.Utilities;

     public bool RemoveUser(string url, string groupName, string userName)  
     {  
       using (ClientContext context = new ClientContext(url))  
       {  
         var principal = Utility.ResolvePrincipal(context, context.Web, userName, PrincipalType.User,  
           PrincipalSource.All, context.Web.SiteUsers, false);  
         context.ExecuteQuery();  
         if (principal.Value != null)  
         {  
           string login = principal.Value.LoginName;  
           GroupCollection siteGroups = context.Web.SiteGroups;  
           Group group = siteGroups.GetByName(groupName);  
           var query = context.LoadQuery(group.Users.Where(usr => usr.LoginName == login).Include(u => u.LoginName));  
           context.ExecuteQuery();  
           User user = query.SingleOrDefault();  
           if (user != null)  
           {  
             group.Users.RemoveByLoginName(user.LoginName);  
             context.ExecuteQuery();  
           }
           return true;               
         }  
       }  
       return false;  
     }  

Add users from other domains to SharePoint

This post should be part of the brain dump series but I never got around to do actually posting it and I sort of needed it last week, at any rate.

A pretty simple command to allow users from other (trusted) domains to be added to SharePoint site:

stsadm -o setproperty -pn peoplepicker-searchadforests -pv "domain:prod.local,prod\ppSP,<password>" -url "https://sp.dev.local"

In our case the problem was that we could not add users from the prod domain, which we use to log in to our machines to the development environment, which we use, well for development :). 

So we created an account in the Prod domain, ppSP, to allow the AD lookups to take place. There needs to be a domain trust between the domains, if there isn't then, you should be looking at claims authentication.

Ms Dynamics CRM 2011 integration with Ms Sharepoint 2010 - Brain Dump 2

A few weeks back we went live and had a couple of issue getting the integration between Dynamics CRM 2011 and SharePoint 2010 working:

1. It would say that the list component was not installed.

I tried following this link to no avail and then reading the comments I realized that I had the same issue as the last comment, namely I was using my account rather than the service account to configure the integration and my account does not have any permissions in SP.

2. I also had this issue.

I followed solution (workaround) number IV.

Get user group membership in Sharepoint from Powershell

A few days back I needed to find out whether the System account (SharePoint\System) was a member of a particular group and it turns out that this cannot be done from the GUI as it doesn't show up in the list of accounts that are members of that group so ... PowerShell to the rescue.

The following snippet will provide a list of users belonging to the desired group <Group Name>:

$site = Get-SPSite -WebApplication "<url>"
$mainsite = (Get-SPWeb -Site $site)[0]
$group = $mainsite.SiteGroups | ?{$_.Name -match "<Group Name>"}
foreach($user in $group.Users){$user}

As usual it needs to be run from the SharePoint PowerShell console or the Sharepoint PowerShell snap in needs to be loaded before running the above commands.

Delete documents from SharePoint document library using PowerShell

So today we had to delete a bunch of documents from SharePoint, so I wrote this script to accomplish this task.

A couple of notes about the script:

1. By default it will not delete anything, just list the urls that contain the $document argument.
2. The $document argument is matched with like as it's matching urls, so be careful that it doesn't delete more documents than you want.

param
(
[string]$url = "https://sp.dev.local",
[string]$document = "Welcome"
[bool]$delete = false
)

$site = Get-SPSite $url

foreach($web in $site.AllWebs) {
    foreach($list in $web.Lists) {
              if($list.BaseType -eq "DocumentLibrary") {
                     foreach($item in $list.Items){
                           if($item.url -like "*$document*")
                           {
                                  if($delete)
                                  {
                                   Write-Host "Deleting $item.url"
                                   $item.File.Delete()
                                  }
                                  else
                                  {
                                   Write-Host "$item.url"
                                  }
                         }               
                      }
              }
       }
}

 

Deploy SharePoint Event Receivers from PowerShell

I thought I would share a script that we use for deploying Event Receivers to SharePoint.

param ([string] $url, [string] $featureName,[string] $solution, [bool] $install, [bool] $uninstall)

if ( -not ($url))
{
 Write-Host "Please enter the Site Url"
 exit
}

function SelectOperation
{
 $message = "Select Operation?";
 $InstallMe = new-Object System.Management.Automation.Host.ChoiceDescription "&Install","Install";
 $UninstallMe = new-Object System.Management.Automation.Host.ChoiceDescription "&Uninstall","Uninstall";
 $choices = [System.Management.Automation.Host.ChoiceDescription[]]($InstallMe,$UninstallMe);
 $answer = $host.ui.PromptForChoice($caption,$message,$choices,0)
 return $answer
}

if (-not($install) -and -not($uninstall))
{
 $answer = SelectOperation

 switch ([int]$answer)
 {
   0 {$install=$true}
   1 {$uninstall=$true}
 }

}

if ($install)
{
 Add-SPSolution -LiteralPath $(join-path $(pwd).Path $solution)
 $solutionId = (Get-SPSolution | ? {$_.name -eq  $solution}).Id
 Install-SPSolution -Identity $solutionId -GACDeployment
 Write-Host "Waiting for the SharePoint to finish..."
 Sleep(120)
 $featureNameId = Get-SPfeatureName | where {$_.displayname -eq  $featureName} 
 Enable-SPfeatureName -Identity $featureNameId -Url $url
}

if ($uninstall)
{
 $featureNameId = (Get-SPfeatureName | where {$_.displayname -eq  $featureName}).Id
 Disable-SPfeatureName $featureNameId -Url $url
 $solutionId = (Get-SPSolution | ? {$_.name -eq  $solution}).Id
 Uninstall-SPSolution -Identity $solutionId
 Write-Host "Waiting for SharePoint to finish..."
 Sleep(120)
 Remove-SPSolution -Identity $solutionId
}
Write-host "All Done."

An example of usage:

.\DeployEV.ps1 -featureName "receiver1" -solution "receiver1.wsp"  -install $true

 Note that the script assumes that the wsp file will be in the same location as the script.

Updates are currently disallowed on GET requests. To allow updates on a GET, set the 'AllowUnsafeUpdates' property on SPWeb.

So today I hit a limit on SharePoint when listing from a library

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator

The solution is simple, just increase the number of items, so from the Central Administration Site:

1. Application Management -> Manage Web Application and select your web application
2. In the Ribbon, click on General Settings drop-down and choose “Resource Throttling”.
3. In the “List View Threshold”, increase the value

The problem I was having was that when I tried to do this I would get the following error:

Updates are currently disallowed on GET requests.  To allow updates on a GET, set the 'AllowUnsafeUpdates' property on SPWeb.

The solution, from the SharePoint PowerShell console:

$sp = get-spwebapplication https://myapp
$sp.HttpThrottleSettings
$sp.Update()

The problem seems to be related to the web application not having a value for HttpThrottleSettings, which will be set by running the above commands.

Issues with Word Automation Services in SharePoint 2010

This week we had an issue with Word Automation Services, in one of our test servers, where our custom code (really boiler plate code, see below) would fail on the second line:

var context = SPServiceContext.GetContext(SPsite);
var wsaProxy = (WordServiceApplicationProxy)context.GetDefaultProxy(typeof(WordServiceApplicationProxy));

Since the same code was working fine in our development environment, it was clear that it was not the code that was at fault but our SharePoint configuration that was at fault.

The issue was that the Word Automation Services Application had not been configured to be added to the default proxy list, see screenshot below, and thus the code was failing to get the proxy.

Note that this adding Word Automation services is done from the Central Administration website:
Central Administration -> Manage Service Applications -> New Word Automation Service

Use AppPool (Application Pool) credentials for Sharepoint integration

I had some fun today trying to work out how to use the AppPool Credentials for integration between our website and SharePoint.

So this is what I did.

using System.Security.Principal;

public class ImpersonateUser : IDisposable
{
  WindowsImpersonationContext ctx = null;
  WindowsImpersonationContext appPoolctx = null;

  public void ImpersonateAppPoolUser()
  {
     //RevertToSelf
     ctx = WindowsIdentity.Impersonate(IntPtr.Zero);
     // and call impersonate on the app pool user object
     appPoolctx = WindowsIdentity.GetCurrent().Impersonate();
  }

  public void UndoUserImpersonation()
  {
     if (ctx != null)
     {
        ctx.Undo();
        ctx = null;
     }

     if (appPoolctx != null)
     {
        appPoolctx.Undo();
        appPoolctx = null;
     }

  }

  public void Dispose()
  {
      UndoUserImpersonation();
  }
}

I used this class for SharePoint integration, as it seemed a quicker and simpler option than storing credential in the web.config and then encrypting.

using (ImpersonateUser ie = new ImpersonateUser())
{
    ie.ImpersonateAppPoolUser();

    using (ClientContext clientContext = new ClientContext(url))
    {
     //Get documents from SharePoint
    }
}

Set Regional Settings for Sharepoint 2010 Sites

Somebody at work forgot to change the regional settings to UK and since we have quite a few sites on our Sharepoint installation, I thought it would be a good idea to script the procedure rather than do it manually, so here it is:

The script takes a single parameter, which is the SharePoint base Url and sets all sites within to UK regional settings, change the GetCultureInfo line to your Locale.

 param ($url) 

 $snapin ="Microsoft.sharepoint.powershell" 

 if (-not $url) 
 { 
 write-host "Please Invoke script like this:  SetRegionalSettings.ps1 -url `"http://sp.dev.com/`"" 
 exit 
 } 

 if ( (Get-PSSnapin -Name $snapin -ErrorAction SilentlyContinue) -eq $null ) 
 {     
  Add-PSSnapin $snapin -ErrorAction SilentlyContinue 
 } 
 
 try 
 { 
  $sites = Get-SPweb -site $url -limit all 
 } 
 catch 
 { 
  Write-Error "An error occurred while retrieving the sharepoint sites for $url" 
 } 
 if ($sites -ne $null) 
 { 
  foreach ($site in $sites) 
  { 
   try 
    { 
     $site.Locale = [System.Globalization.CultureInfo]::GetCultureInfo("en-GB");    
     Write-Host "Changing Regional Settings for Site $($site.Name) to $($site.Locale.DisplayName)"   
     $site.Update() 
    } 
    catch  
    { 
     Write-Error "An error occurred while changing regional settings" 
    } 
  } 
 } 

What the ...., Microsoft?

In the run up to Christmas we seem to have a very light workload and I thought I'd give MS Sharepoint a try to see what all the fuss is about.

I got the ISO from MSDN and after using my newly acquired mad PowerCLI skills to clone a new VM, I ran the Sharepoint installer.

I clicked on the install pre-requisites option and after a few minutes it stopped working, why? because it could not connect to the internet, say what?

You expect an enterprise grade application to have internet access? This is insane, this is the sort of crap that really makes me wish I had nothing to do with Microsoft.

Anyway, you can find all the pre-requisites here, download them and then you can install Sharepoint 2010 SP1, maybe.

What I really don't understand is why they are not contained in the ISO: if they are pre-requisites then they should be in the ISO. It's only the optional pre-requisites, that's a good oxymoron, that should need to be downloaded, but mandatory pre-requisites? 

I sometimes don't understand Microsoft. It seems to bend over backwards to make life easier for everybody, particularly developers, and every so often you get stuff like this, which makes no sense whatsoever.