Wednesday, 3 February 2016

MSDN Subscription - A Very British Rip-Off

So a few days back I was considering whether we should take the plunge and fork out for an MSDN subscription for all our developers, there is five of us and given where we want to be it made sense.

This was a cost that had been budgeted for so, in theory, this should not be a problem, but as with all large purchases, and for a small firm, 5 MSDN subscriptions is a Large purchase, which meant that we had to get approval from above so I went to this site to confirm prices.


5 x $1199 even at silly exchange rates, should be less than £5K, so I thought, this would be an easy sell as it was less than we had budgeted, boy was I wrong.

If you are in the UK, you can see for yourself here, which shows you this:


What????? 

The exchange rate used by Microsoft is $1 = £1.01, rather than $1 = £0.68, so close to 50% surcharge. 

To put it another way, if we converted back to dollars, the price would be ~ $1780 rather than $1199.

I really like to hear an explanation from Microsoft, it certainly can't be the taxes the have to pay in this country, as the sales will probably be billed to the Irish subsidiary.

For the record this is the exchange rate today, according to Google:



and for the really lazy: 


I know there are resellers out there that do a cheaper price, it still does not change the fact that Microsoft itself applies an almost 50% surcharge to UK developers.

Sunday, 24 January 2016

Migrating from Mercurial Kiln to Visual Studio Online (Git)

So we migrated all of our Mercury repositories, hosted in Kiln, to Visual Studio Online last week and this is the procedure we followed.

We only had 3 separate repositories and they are relatively small, so we deemed it unnecessary to try a more scalable approach. 

All the command line commands are run from a Developer Command Prompt for VS 2015.

1. Clone Mercury Repository to be migrated to a new directory.
hg clone <repo url>
2. Create new Harmony Repository
a. Navigate to https://<yourcompany>.kilnhg.com/Code/Repositories
b. Click on New Repository

It's very important that you select a Harmony Repository, otherwise this process will not work. Forking or Branching a Mercury only repository will not work either, as it simple creates a new Mercury repository.
3. Push to new Harmony Repository
 a. Edit  the .hg/hgrc file and change the path the new repository url
[paths]
default = https://<>.kilnhg.com/Code/Repositories/Group/Test_Migration
b. hg push  
Once this is has finished the Test_Migration repository is a Git & Mercury repository of the initial Mercury only repo, so it's relatively simple from this point onward.
4. Clone Harmony Repository as a Git Repository
git clone <https://<>.kilnhg.com/Code/Repositories/Group/Test_Migration.git>
5. Create new Repository in VSO
a. Navigate to Project
b. Click on Code
Once created you can click on Explorer which will take you to this page

6. Push to VSO
git remote remove origin
git remote add origin https://<yourcompany>.visualstudio.com/DefaultCollection/<project>/_git/<repo>
git push -u origin --all

Sunday, 10 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 6

In the last post in the series, we used a DTO to minimize the transfered data but we retrieved all that data from the database, shouldn't we have got just what we wanted?

In short the answer is yes, again the potential savings need to be carefully considered as the code might be more cluttered.

This is the query that gets run against the database:

SELECT [x].[SubRedditId], [x].[IsBanned], [x].[IsDefault], [x].[IsPublic], [x].[Name]
FROM [SubReddit] AS [x]
WHERE ([x].[IsDefault] = 1 AND [x].[IsPublic] = 1) AND [x].[IsBanned] = 0
You might be tempted to modify the Repository method like this:
public IEnumerable<SubRedditDTO> GetDefaultSubReddits()
{
            return ctx.SubReddits
                .Where(x => x.IsDefault && x.IsPublic && !x.IsBanned)
                .Cast<SubRedditDTO>();       
}
If you do this, you'll get this error: System.InvalidOperationException:
No coercion operator is defined between types 'Reddit.Models.SubReddit' and 'Reddit.Models.SubRedditDTO'.
We could use AutoMapper here but it seems a bit clunky when you're trying to reduce object size as the definition needs to be a bit like this:
          Mapper.Initialize(config =>
            {
                config.CreateMap<SubReddit, SubRedditDTO>()
                .ForSourceMember(x => x.IsBanned, o => o.Ignore());               
            });
In all honesty I've not read enough about AutoMapping, DTOs and ViewModels to be able to advise on best practices, but in this case it does seem a bit clunky, so instead I tried the following:
       public IEnumerable<SubRedditDTO> GetDefaultSubReddits()
        {
            return ctx.SubReddits
                .Where(x => x.IsDefault && x.IsPublic && !x.IsBanned)
                .Select(x => new SubRedditDTO() { Name=x.Name });
        }
This actually achieves what we wanted, at least in theory, which is reduce the load on the database. I realize that DB performance is a lot more nuanced than I'm presenting it here, but as a rule of thumb the less data retrieved from the database the better. This is the query that is executed against the database:
SELECT [x].[Name]
FROM [SubReddit] AS [x]
WHERE ([x].[IsDefault] = 1 AND [x].[IsPublic] = 1) AND [x].[IsBanned] = 0
This should be better from a DB performance point of view, but how much, it's hard to say.

The last thing I'll tackle in this post is the lack of links on the front page. You might be tempted to simply change the Index.cshtml page to this:

@{
    ViewData["Title"] = "Reddit";
}
@section scripts {
    <script src="~/lib/angular/angular.js"></script>
    <script src="~/js/redditapp.js"></script>
}
<div ng-app="redditApp">
    <div ng-controller="SubRedditListCtrl">
        <ul class="list-inline">
            <li ng-repeat="subReddit in subReddits">
                <a href="http://reddit.com/r/{{subReddit.Name}}">{{subReddit.Name}}</a>                      
            </li>
        </ul>
    </div>
    <hr />
</div>
Unfortunately, that will not always work as expected, all links will not have the subreddit until angular has substituted the value of subReddit.Name, instead we need to use a angular directive:

<a ng-href="http://reddit.com/r/{{subReddit.Name}}">{{subReddit.Name}}</a>  

Sunday, 3 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 5

In the last post we finally started to do something to the front end, namely the SubReddit bar on top, but the values were hard coded and they weren't even links.

Let's try to remedy that.

There are a few things we need to do:
  • How do we identify Default SubReddits?
  • How do we get and display said SubReddits?
The first one is easy, we add a new property to our SubReddit class, which now looks like this:
using System.Collections.Generic;

namespace Reddit.Models
{
    public class SubReddit
    {
        public int SubRedditId { get; set; }
        public string Name { get; set; }
        public bool IsPublic { get; set; }
        public List<Post> Posts { get; set; }
        public List<RedditUser> Users { get; set; }
        public bool IsBanned { get; set; }
        public bool IsDefault { get; set; }
    }
}

There are other ways of doing this, say, have a default SubReddit table an add the default SubReddits there, but we'll do it this way until I change my mind.

We need to run now through the Migrations rigmarole again, so from ..\src\Reddit run the following commands:
dnx ef Migrations add isDefaultSubReddit
dnx ef database update
I've manually seeded the SubReddit table with a few records, both default and not default (I will discuss how to seed it from code in a later post) and now we can edit redditApp.js to this:
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', ['$scope', '$http', function($scope, $http) {
  $http.get("/api/SubReddit").then(function(data){
  $scope.subReddits =data.data ;
})}
])
  
})();

In order to get around issues with minification, we inject the dependencies as strings which will not be minified.

There are several issues with this approach as we're getting more data than we need in two senses:

  • We're getting all SubReddits, we could filter them but that would mean retrieving all the data from the server and then filtering them, which is as inefficient as it sounds. 
  • Furthermore, we're retrieving all properties of the SubReddit objects when we only need the name.
Let's deal with the first of these issues. We add a new method called GetDefaultSubReddits to the Repository, make sure you add it to the Interface as well:

        public IEnumerable<SubReddit> GetDefaultReddits()
        {
            return ctx.SubReddits.Where(x => x.IsDefault && x.IsPublic && !x.IsBanned);
        }
And we also add a call to this method from the Controller:
.        [HttpGet("[action]")]
        public IEnumerable<SubReddit> GetDefault()
        {
            return repository.GetDefaultSubReddits();
        }
Note that I'm using an action here, which is effectively the name of the method
Now we modify the redditApp.js
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', ['$scope', '$http', function($scope, $http) {
  $http.get("/api/SubReddit/GetDefault").then(function(data){
  $scope.subReddits =data.data ;
})}
])
  
})();
At this point we've solved issue number one but we're still sending quite a bit of unnecessary data up and down the wire so let's build a DTO that contains just the data that we need, which in this case is simply the Name. There are other ways of achieving this, for instance we could use an OData query to simply retrieve the necessary data.

This is the SubRedditDTO

namespace Reddit.Models
{
    public class SubRedditDTO
    {
        public string Name { get; set; }
    }
}
Modified GetDefault Method, I'm not sure whether it's a better idea to transform to DTO object here or on the Repository. I guess it depends on usage, in other words, if the Repository method will be used somewhere else it makes sense to do the transformation here.
        [HttpGet("[action]")]
        public IEnumerable<SubRedditDTO> GetDefault()
        {
            return repository.GetDefaultSubReddits().Select(x => new SubRedditDTO() { Name = x.Name });
        }
This is the reponse from the server now
[{"Name":"Gadgets"},{"Name":"Sports"},{"Name":"Gaming"},{"Name":"Pics"},{"Name":"WorldNews"},{"Name":"Videos"},{"Name":"AskReddit"},{"Name":"Aww"}]
This is before, when we where not using a DTO.
[{"SubRedditId":3,"Name":"Gadgets","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":4,"Name":"Sports","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":5,"Name":"Gaming","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":6,"Name":"Pics","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":7,"Name":"WorldNews","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":8,"Name":"Videos","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":9,"Name":"AskReddit","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":10,"Name":"Aww","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true}]
This might seem like a paltry gain but little savings like this can quickly add up.

Saturday, 2 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 4

In the previous post we ended up again with a broken, although partially this time, application let's try to fix it.

The problem was that it could not resolve the service type for IRedditRepository as we were not telling how to inject the dependency properly, which is done on the ConfigureServices method on Startup.cs:

public void ConfigureServices(IServiceCollection services)
        {
            // Add framework services.
            services.AddMvc();
            services.AddEntityFramework()
                .AddSqlServer()
                .AddDbContext<RedditContext>();

            services.AddScoped<IRedditRepository, RedditRepository>();
        }
AddScoped will use the same instance for each request, could have used AddSingleton or AddInstance.

We try running this again and we see this:


So it's working but there is nothing there, we'll have a look at this later on.

Now let's try to look at the homepage, which at the moment looks like this.

This doesn't look like Reddit, so let's do a bit of editing.

Delete the Contact.cshtml and About.cshtml files, in the Home folder

Edit Index.cshtml so that it looks like this:

@{
    ViewData["Title"] = "Reddit";
}
This is the _Layout.cshtml file, which can be thought of as a master page or template page. I've removed pretty much everything from it that might be rendered.
<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>@ViewData["Title"] - Reddit</title>

    <environment names="Development">
        <link rel="stylesheet" href="~/lib/bootstrap/dist/css/bootstrap.css" />
        <link rel="stylesheet" href="~/css/site.css" />
    </environment>
    <environment names="Staging,Production">
        <link rel="stylesheet" href="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.5/css/bootstrap.min.css"
              asp-fallback-href="~/lib/bootstrap/dist/css/bootstrap.min.css"
              asp-fallback-test-class="sr-only" asp-fallback-test-property="position" asp-fallback-test-value="absolute" />
        <link rel="stylesheet" href="~/css/site.min.css" asp-append-version="true" />
    </environment>
</head>
<body>
    <div>
        @RenderBody()       
    </div>

    <environment names="Development">
        <script src="~/lib/jquery/dist/jquery.js"></script>
        <script src="~/lib/bootstrap/dist/js/bootstrap.js"></script>
        <script src="~/js/site.js" asp-append-version="true"></script>
    </environment>
    <environment names="Staging,Production">
        <script src="https://ajax.aspnetcdn.com/ajax/jquery/jquery-2.1.4.min.js"
                asp-fallback-src="~/lib/jquery/dist/jquery.min.js"
                asp-fallback-test="window.jQuery">
        </script>
        <script src="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.5/bootstrap.min.js"
                asp-fallback-src="~/lib/bootstrap/dist/js/bootstrap.min.js"
                asp-fallback-test="window.jQuery && window.jQuery.fn && window.jQuery.fn.modal">
        </script>
        <script src="~/js/site.min.js" asp-append-version="true"></script>
    </environment>

    @RenderSection("scripts", required: false)
</body>
</html>
The end result of all this should be an empty page.

I've left RenderBody and RenderSection method calls intact as they will come in handy later. In essence, they are used to merge in the content from the actual pages and to load the scripts needed respectively.

Since this is a modern app I'm going to use AngularJS, which is probably past it's prime now but there you go.

The simplest thing to do this is by using the manage bower packages:


This will display this page:


This is the result on the bower.json file, which should be noted is not shown by visual studio but it's there on the src\Reddit folder:

{
  "name": "ASP.NET",
  "private": true,
  "dependencies": {
    "bootstrap": "3.3.5",
    "jquery": "2.1.4",
    "jquery-validation": "1.14.0",
    "jquery-validation-unobtrusive": "3.2.4",
    "angular": "1.4.6"
  }
}
So let's add Angular to Reddit

I've added a new file called redditApp.js to wwwroot/js, which contains the following:
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', function($scope) {
  $scope.subReddits = [
    {'name': 'Gadgets'},    
    {'name': 'Sports'},
    {'name': 'Gaming'},
    {'name': 'Pics'},
  ];
});
  
})();

I've hard coded a few subreddits to have something to display but we will soon get them from the database. In essence, the above is a self executing function containing an extremely simple angular controller that has an array of objects.

I have modified the Index.cshtml like this:

@{
    ViewData["Title"] = "Reddit";
}
@section scripts {

    <script src="~/lib/angular/angular.js"></script>
    <script src="~/js/redditapp.js"></script>
}
<div ng-app="redditApp">
    <div ng-controller="SubRedditListCtrl">
        <ul class="list-inline">
            <li ng-repeat="subReddit in subReddits">
                <span>{{subReddit.name}}</span>
            </li>
        </ul>
    </div>
</div>

Note the angular directives? ng-app, ng-controller and ng-repeat and also note the angular bingind {{subReddit.name}}

This is what Reddit looks like now:

Friday, 1 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 3

In the last post we applied a database migration but nothing occurred.

The reason for this was that the RedditContext class was not tracking any of the objects nor where the relationships specified on the OnModelCreating.

So I've modified the RedditContext class

using Microsoft.Data.Entity;

namespace Reddit.Models
{
    public class RedditContext : DbContext
    {
        public RedditContext()
        {
            //Database.EnsureCreated();
        }

        public DbSet<RedditUser> RedditUsers { get; set; }
        public DbSet<UserDetails> UserDetails { get; set; }
        public DbSet<SubReddit> SubReddits { get; set; }
        public DbSet<Post> Posts { get; set; }
        public DbSet<Comment> Comments { get; set; }
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            var connection = Startup.Configuration["Data:RedditContextConnection"];
            optionsBuilder.UseSqlServer(connection);
            base.OnConfiguring(optionsBuilder);
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<RedditUser>()
                .HasKey(x => x.RedditUserId);

            modelBuilder.Entity<UserDetails>()
                .HasKey(x => x.UserId);

            modelBuilder.Entity<RedditUser>()
                .HasOne(x => x.UserDetails)
                .WithOne(x => x.User);

            base.OnModelCreating(modelBuilder);
        }
    }
}

In order to recover from this mistake, we want to apply the new migration, but the Core migration contains nothing so I will first unapply it
dnx ef database update Start
dnx ef migrations remove
dnx ef migrations add Core
dnx ef database update
This time it worked .... hurray


At this point, it's worth considering how will this be deployed to a test environment and the answer is pretty simple. A migration script can be created with the following command:
dnx ef migrations script
This creates a Sql script that can be used to create the database.

It's time to have another look at our repository
Let's start with the IRedditRepository, where we now have a few methods and this time we'll try to implement some of them.
using System.Collections.Generic;

namespace Reddit.Models
{
    public interface IRedditRepository
    {
        UserDetails GetUserDetails();
        IEnumerable<SubReddit> GetSubscribedSubReddits(RedditUser user);
        IEnumerable<SubReddit> GetSubReddits();
        IEnumerable<Post> GetSubRedditPosts(SubReddit subreddit);
        IEnumerable<Comment> GetPostComments(Post post);
    }
}
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;

namespace Reddit.Models
{
    public class RedditRepository : IRedditRepository
    {
        private readonly RedditContext ctx;
        private readonly ILogger<RedditRepository> logger;

        public RedditRepository(RedditContext ctx, ILogger<RedditRepository> logger)
        {
            this.ctx = ctx;
            this.logger = logger;
        }

        public IEnumerable<Comment> GetPostComments(Post post)
        {
            return ctx.Posts
               .Where(x => x.PostId == post.PostId)
               .Single()
               .Comments;
        }

        public IEnumerable<Post> GetSubRedditPosts(SubReddit subreddit)
        {
            return ctx.SubReddits
                .Where(x => x.SubRedditId == subreddit.SubRedditId)
                .Single()
                .Posts;
        }

        public IEnumerable<SubReddit> GetSubReddits()
        {
            return ctx.SubReddits.AsEnumerable();
        }

        public IEnumerable<SubReddit> GetSubscribedSubReddits(RedditUser user)
        {
            throw new NotImplementedException();
        }

        public UserDetails GetUserDetails()
        {
            throw new NotImplementedException();
        }
    }
}

Straight away we see that there are going to be problems as we lack paging capabilities, but we'll add those later

In any case, we'll need to expose these method somehow, there are potentially two options here depending on what we're trying to accomplish: 

We could use these methods from inside a Controller or We could create an API and we're definitely going for the latter as it's way cooler

Create a new folder called Api inside the Controllers folder and add a new  Web API controller class, called SubRedditController
using System.Collections.Generic;
using Microsoft.AspNet.Mvc;
using Reddit.Models;

// For more information on enabling Web API for empty projects, visit http://go.microsoft.com/fwlink/?LinkID=397860

namespace Reddit.Controllers.Api
{
    [Route("api/[controller]")]
    public class SubRedditController : Controller
    {
        private readonly IRedditRepository repository;

        public SubRedditController(IRedditRepository repository)
        {
            this.repository = repository;
        }

        // GET: api/values
        [HttpGet]
        public IEnumerable<SubReddit> Get()
        {
            return repository.GetSubReddits();
        }

    }
}

We can now build and debug the project. This allow us to navigate to http://localhost:<portnumber>/api/SubReddit

Unfortunately this does not work:


We'll look at the solution to this problem in the next post.

Thursday, 31 December 2015

Cloning Reddit to learn ASP.NET 5 - Part 2

In the previous post we got started in our quest to build a Reddit Clone. We ended up with a broken project so let's try to fix that.

The first thing is to add a UserDetails class and then link it to a RedditUser class, which will also need to be added to the Models folder.

UserDetails.cs
namespace Reddit.Models
{
    public class UserDetails
    {
        public int UserId { get; set; }
        public long LinkKarma { get; set; }
        public long CommentKarma { get; set; }
        public RedditUser User { get; set; }      
    }
}

RedditUser.cs
namespace Reddit.Models
{
    public class RedditUser
    {
        public int RedditUserId { get; set; }
        public string Nick { get; set; }
        public UserDetails UserDetails { get; set; }
    }
}

We will further modify the RedditUser class to allow users to login, that's the reason for the rather small class. If this were a real clone then we'd probably want to use Guids for the Keys.

At this point we should have a working project again, which can be verified by building and/or debugging it.

We now want to create the database with the RedditUser and UserDetails and the right relationships between them, namely a 1 to 1 relationship.

Firstly, modify the Startup.cs file so that the Configuration property is static, which will allow us access to this property without having to instantiate the class again and to add the EntityFramework service.
public static IConfigurationRoot Configuration { get; set; }

public void ConfigureServices(IServiceCollection services)
        {
            // Add framework services.
            services.AddMvc();
            services.AddEntityFramework()
                .AddSqlServer()
                .AddDbContext<RedditContext>();      
        }
Edit the appsettings.json file
 "Data": {
    "RedditContextConnection": "Server=(localdb)\\MSSQLLocalDB;Database=Reddit;Trusted_Connection=true;MultipleActiveResultSets=true"
  }
Finally let's edit the Context class (RedditContext.cs)
using Microsoft.Data.Entity;

namespace Reddit.Models
{
    public class RedditContext : DbContext
    {
        public RedditContext()
        {
            //Database.EnsureCreated();
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            var connection = Startup.Configuration["Data:RedditContextConnection"];
            optionsBuilder.UseSqlServer(connection);
            base.OnConfiguring(optionsBuilder);          
        } 

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<RedditUser>()
                .HasKey(x => x.RedditUserId);

            modelBuilder.Entity<UserDetails>()
                .HasKey(x => x.UserId);
            
            modelBuilder.Entity<RedditUser>()
                .HasOne(x => x.UserDetails)
                .WithOne(x => x.User);

            base.OnModelCreating(modelBuilder);
        }
    }
}

The OnConfiguring method sets up the connection String to the database.

The OnModelCreating method sets up the relationship between the tables in the database, it's worth mentioning here that I'm using the Fluent API to do this, but it's also possible to do it via data annotations.

I'm torn as to what the best way of doing this is, as both have advantages and disadvantages. I suspect that I will stick to the this syntax for the time being, even if I probably won't need any of the features that it provides over and above data annotations

I have added explicitly added the Primary Keys for each table, this is only necessary for UserDetails as the Key is different to the table name (I think) Now we can add create the database, from src\Reddit run the following command, which will create a Migration called Start:
 dnx ef migrations add Start 
Note that a new folder called Migrations will be created and populated with two files: A migration file, normally named DateTime_MigrationName and a model Snapshot.

When I first run this, it created the database. This was due to the commented out logic in the constructor, which wasn't commented out, once it was commented out, the expected behaviour resumed and we need to run the following command to apply the changes:
dnx ef database update
Let's add a few more models for SubReddit, Posts and Comments:

using System.Collections.Generic;

namespace Reddit.Models
{
    public class SubReddit
    {
        public int SubRedditId { get; set; }
        public string Name { get; set; }
        public bool IsPublic { get; set; }
        public List<Post> Posts { get; set; }
        public List<RedditUser> { get; set; }
        public bool IsBanned { get; set; }

    }
}
using System.Collections.Generic;

namespace Reddit.Models
{
    public class Post 
    {
        public int PostId { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }
        public string Link { get; set; }
        public int UpVotes { get; set; }
        public int DownVotes { get; set; }
        public List<Comment> Comments { get; set; }
        public RedditUser Author { get; set; }
        public int AuthorId { get; set; }
        public SubReddit SubReddit { get; set; }
        public int SubRedditId { get; set; }


    }
}
namespace Reddit.Models
{
    public class Comment
    {
        public int CommentId { get; set; }
        public string Title { get; set; }
        public string Text { get; set; }
        public int UpVotes { get; set; }
        public int DownVotes { get; set; }
        public RedditUser Author { get; set; }
        public int AuthorId { get; set; }
    }
}
I have added Foreign Keys to Comment and Post to make querying easier. We create another migration called Core and apply it
dnx ef migrations add Core
dnx ef database update
Oddly nothing happens, why?

Wednesday, 30 December 2015

Cloning Reddit to learn ASP.NET 5 - Part 1

I've always found it a bit boring to follow a book or video to try to learn a new, or old technology, so I thought I would try a different approach this time.

I would set myself a project, which while perhaps not that much better than a Bicycle store at least it would be more interesting for me.

So I'm doing a clone of Reddit...obviously it won't be a fully functional clone of Reddit but it should have most of the functionality, which will create some interesting challenges.

I don't know how many posts there will be as I am making it up as I go along.

DISCLAIMER

I'm using this as a learning experience, so there is likely a lot that will be wrong, which I will try to amend when I realize that it is wrong so please bear that in mind if you're reading this. It also means that some things might get changed dramatically but I guess that's part of the learning process, right?

Pre-Requisites

  • Visual Studio 2015 (A free edition can be downloaded from this page)
  • ASP.NET 5 RC 1, which can be downloaded from here

We start by creating a new Web Application Project in Visual Studio, which we'll call Reddit:


Ensure that you select Web Application but rather than use the Out of the Box Authentication we'll use no authentication.



We are going to need to use a database to store all the data, controversially, I'm not using a NOSQL DB, although EF7 will support them

Let's add Entity Framework to the project.

We have to edit the project.json file to do this. The added parts in bold. 
"dependencies": {
    "EntityFramework.Core": "7.0.0-rc1-final",
    "EntityFramework.Commands": "7.0.0-rc1-final",
    "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
    "Microsoft.AspNet.Diagnostics": "1.0.0-rc1-final",
    "Microsoft.AspNet.IISPlatformHandler": "1.0.0-rc1-final",
    "Microsoft.AspNet.Mvc": "6.0.0-rc1-final",
    "Microsoft.AspNet.Mvc.TagHelpers": "6.0.0-rc1-final",
    "Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
    "Microsoft.AspNet.StaticFiles": "1.0.0-rc1-final",
    "Microsoft.AspNet.Tooling.Razor": "1.0.0-rc1-final",
    "Microsoft.Extensions.Configuration.FileProviderExtensions": "1.0.0-rc1-final",
    "Microsoft.Extensions.Configuration.Json": "1.0.0-rc1-final",
    "Microsoft.Extensions.Logging": "1.0.0-rc1-final",
    "Microsoft.Extensions.Logging.Console": "1.0.0-rc1-final",
    "Microsoft.Extensions.Logging.Debug": "1.0.0-rc1-final",
    "Microsoft.VisualStudio.Web.BrowserLink.Loader": "14.0.0-rc1-final"
  },

  "commands": {
    "web": "Microsoft.AspNet.Server.Kestrel",
    "ef": "EntityFramework.Commands"
  },

This will pull the relevant DLLs into the project. It's worth pointing out that nuget will let you choose which version to use, so there is no need to memorize the version numbers :)

At this point we can use database migrations but we don't have any models yet... IF you've used EF 6, Migrations are done differently for EF 7. In essence, the command is (note that it needs to be run from the src folder):
dnx ef <command> <options>
Before adding any models, we need to be able to access the database.  First, let's add a folder called Models and then a simple Repository Interface that I'll call IRedditRepository, along with an implementation RedditRepository and a RedditContext class, which is the actual DB Context.

RedditContext.cs
using Microsoft.Data.Entity;

namespace Reddit.Models
{
    public class RedditContext : DbContext
    {
        public RedditContext()
        {
            Database.EnsureCreated();
        }
    }
}
IRedditRepository.cs

namespace Reddit.Models
{
    public interface IRedditRepository
    {
        UserDetails GetUserDetails();
    }
}
UserDetails is a Data Model and will be defined in the next post, so don't worry too much about these yet.

RedditRepository.cs

using Microsoft.Extensions.Logging;
using System;

namespace Reddit.Models
{
    public class RedditRepository : IRedditRepository
    {
        private readonly RedditContext ctx;
        private readonly ILogger<RedditRepository> logger;

        public RedditRepository(RedditContext ctx, ILogger<RedditRepository> logger)
        {
            this.ctx = ctx;
            this.logger = logger;
        }

        public UserDetails GetUserDetails()
        {
            throw new NotImplementedException();
        }      
    }
}
Other than having a broken project we've not achieved much, yet :(

Stayed tuned for part 2.

Friday, 20 November 2015

Instantiating the OrganizationServiceProxy for IFD enabled organizations.

Last week was an interesting week at work, highlights include running an application from my laptop against the live service.

It took me a while to get this working because I was using the wrong endpoint, D'oh, and the wrong type of credentials, double D'oh.

In any case, here it is for posterity:
public class IFDCrmService

    {
        const string ServiceUrl = "https://{0}/XRMServices/2011/Organization.svc";

        public static IOrganizationService GetIFDCrmService()
        {
            ClientCredentials credentials = new ClientCredentials();
            credentials.UserName.UserName = ConfigurationManager.AppSettings["UserName"];
            credentials.UserName.Password = ConfigurationManager.AppSettings["Password"];

            if (ConfigurationManager.AppSettings["FQDN"] == null) { throw new ArgumentNullException("FQDN key missing."); }
            string fqdn = ConfigurationManager.AppSettings["FQDN"]; 
        
            return new OrganizationServiceProxy(new Uri(string.Format(ServiceUrl, fqdn)), null, credentials, null);
        }
    }
The relevant part of the app.config is below:
<appSettings>
    <add key="UserName" value="dev\crmapppool" />
    <add key="Password" value="P@55w0rd1" />    
    <add key="FQDN" value="crmdev.dev.com"/>
 </appSettings> 
It's always a good idea to encrypt the passwords but I won't discuss this today here.

Monday, 26 October 2015

Adventures using Availability Groups and RBS with SharePoint 2013

The concept behind a remote blob storage is pretty simple, see this for instance. I just want to talk about the myriad issues we've had when using RBS. with availability groups.

Our database setup uses Availability Groups, which, and this is controversial, is a cheap cluster. I do get that there are advantages to availability groups but these seem to be outweighed by the disadvantages. I know this is just my opinion and that I also know nothing about availability groups, HA clusters or anything in general, thank you for pointing out.

So what are the drawbacks of AGs?

  • Official Support is patchy. e.g. in Dynamics CRM 2013 one is forced to update the database directly.
  • Performance can be an issue as the database is always using at least two boxes.
  • Stuff flat out refuses to work, e.g. RBS Maintainer, various SharePoint database related operations.
To the AG mix we introduced RBS and this is were things started to go horribly wrong for us.

The first issue we encountered was the inability to delete a content database from SharePoint, which is not a major issue but it's really annoying.

The second issue was that the RBS maintianer would not work, so the storage requirements would just keep growing. This might not be an issue if you don't plan to archive your documents, but our DB had ~500GB of docs, about 2/3 of which were old but for contractual reasons needed to be kept.

This effectively put a nail in the coffin of the RBS + AG combo but there is more.

In order to load the ~500 GB document, we had a tool to upload the documents to SharePoint. This tool was multi-threaded and it essentially read the documents from the source DB and uploaded them to SharePoint, using the SharePoint CSOM model.

At this point, it's worth mentioning that our hosting provider does not guarantee any sort of performance level, too long to explain.

A couple of weeks back, with RBS on the database, we did a trial run of the upload and we were hitting very poor rates, ~ 4 GB per hour.

Last week, after RBS had been disabled and the content databases recreated, we tried a second trial run and the speed jumped to ~ 20 GB per hour.

I can't say that our RBS configuration was perfect, I think the threshold was on the low side (128 KB) but even so, the speed increase has been massive.

It actually gets better, because the 4 GB per hour figure was using both servers in the farm, whereas the 20 GB per hour figure was simply using one.

yes, yes, I know our hosting provider is crap and 128 KB is below the recommendation, but a 5 fold increase in transfer rates and a lowering of the error rate to almost zero is something that should be considered.