Sunday 24 January 2016

Migrating from Mercurial Kiln to Visual Studio Online (Git)

So we migrated all of our Mercury repositories, hosted in Kiln, to Visual Studio Online last week and this is the procedure we followed.

We only had 3 separate repositories and they are relatively small, so we deemed it unnecessary to try a more scalable approach. 

All the command line commands are run from a Developer Command Prompt for VS 2015.

1. Clone Mercury Repository to be migrated to a new directory.
hg clone <repo url>
2. Create new Harmony Repository
a. Navigate to https://<yourcompany>.kilnhg.com/Code/Repositories
b. Click on New Repository

It's very important that you select a Harmony Repository, otherwise this process will not work. Forking or Branching a Mercury only repository will not work either, as it simple creates a new Mercury repository.
3. Push to new Harmony Repository
 a. Edit  the .hg/hgrc file and change the path the new repository url
[paths]
default = https://<>.kilnhg.com/Code/Repositories/Group/Test_Migration
b. hg push  
Once this is has finished the Test_Migration repository is a Git & Mercury repository of the initial Mercury only repo, so it's relatively simple from this point onward.
4. Clone Harmony Repository as a Git Repository
git clone <https://<>.kilnhg.com/Code/Repositories/Group/Test_Migration.git>
5. Create new Repository in VSO
a. Navigate to Project
b. Click on Code
Once created you can click on Explorer which will take you to this page

6. Push to VSO
git remote remove origin
git remote add origin https://<yourcompany>.visualstudio.com/DefaultCollection/<project>/_git/<repo>
git push -u origin --all

Sunday 10 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 6

In the last post in the series, we used a DTO to minimize the transfered data but we retrieved all that data from the database, shouldn't we have got just what we wanted?

In short the answer is yes, again the potential savings need to be carefully considered as the code might be more cluttered.

This is the query that gets run against the database:

SELECT [x].[SubRedditId], [x].[IsBanned], [x].[IsDefault], [x].[IsPublic], [x].[Name]
FROM [SubReddit] AS [x]
WHERE ([x].[IsDefault] = 1 AND [x].[IsPublic] = 1) AND [x].[IsBanned] = 0
You might be tempted to modify the Repository method like this:
public IEnumerable<SubRedditDTO> GetDefaultSubReddits()
{
            return ctx.SubReddits
                .Where(x => x.IsDefault && x.IsPublic && !x.IsBanned)
                .Cast<SubRedditDTO>();       
}
If you do this, you'll get this error: System.InvalidOperationException:
No coercion operator is defined between types 'Reddit.Models.SubReddit' and 'Reddit.Models.SubRedditDTO'.
We could use AutoMapper here but it seems a bit clunky when you're trying to reduce object size as the definition needs to be a bit like this:
          Mapper.Initialize(config =>
            {
                config.CreateMap<SubReddit, SubRedditDTO>()
                .ForSourceMember(x => x.IsBanned, o => o.Ignore());               
            });
In all honesty I've not read enough about AutoMapping, DTOs and ViewModels to be able to advise on best practices, but in this case it does seem a bit clunky, so instead I tried the following:
       public IEnumerable<SubRedditDTO> GetDefaultSubReddits()
        {
            return ctx.SubReddits
                .Where(x => x.IsDefault && x.IsPublic && !x.IsBanned)
                .Select(x => new SubRedditDTO() { Name=x.Name });
        }
This actually achieves what we wanted, at least in theory, which is reduce the load on the database. I realize that DB performance is a lot more nuanced than I'm presenting it here, but as a rule of thumb the less data retrieved from the database the better. This is the query that is executed against the database:
SELECT [x].[Name]
FROM [SubReddit] AS [x]
WHERE ([x].[IsDefault] = 1 AND [x].[IsPublic] = 1) AND [x].[IsBanned] = 0
This should be better from a DB performance point of view, but how much, it's hard to say.

The last thing I'll tackle in this post is the lack of links on the front page. You might be tempted to simply change the Index.cshtml page to this:

@{
    ViewData["Title"] = "Reddit";
}
@section scripts {
    <script src="~/lib/angular/angular.js"></script>
    <script src="~/js/redditapp.js"></script>
}
<div ng-app="redditApp">
    <div ng-controller="SubRedditListCtrl">
        <ul class="list-inline">
            <li ng-repeat="subReddit in subReddits">
                <a href="http://reddit.com/r/{{subReddit.Name}}">{{subReddit.Name}}</a>                      
            </li>
        </ul>
    </div>
    <hr />
</div>
Unfortunately, that will not always work as expected, all links will not have the subreddit until angular has substituted the value of subReddit.Name, instead we need to use a angular directive:

<a ng-href="http://reddit.com/r/{{subReddit.Name}}">{{subReddit.Name}}</a>  

Sunday 3 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 5

In the last post we finally started to do something to the front end, namely the SubReddit bar on top, but the values were hard coded and they weren't even links.

Let's try to remedy that.

There are a few things we need to do:
  • How do we identify Default SubReddits?
  • How do we get and display said SubReddits?
The first one is easy, we add a new property to our SubReddit class, which now looks like this:
using System.Collections.Generic;

namespace Reddit.Models
{
    public class SubReddit
    {
        public int SubRedditId { get; set; }
        public string Name { get; set; }
        public bool IsPublic { get; set; }
        public List<Post> Posts { get; set; }
        public List<RedditUser> Users { get; set; }
        public bool IsBanned { get; set; }
        public bool IsDefault { get; set; }
    }
}

There are other ways of doing this, say, have a default SubReddit table an add the default SubReddits there, but we'll do it this way until I change my mind.

We need to run now through the Migrations rigmarole again, so from ..\src\Reddit run the following commands:
dnx ef Migrations add isDefaultSubReddit
dnx ef database update
I've manually seeded the SubReddit table with a few records, both default and not default (I will discuss how to seed it from code in a later post) and now we can edit redditApp.js to this:
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', ['$scope', '$http', function($scope, $http) {
  $http.get("/api/SubReddit").then(function(data){
  $scope.subReddits =data.data ;
})}
])
  
})();

In order to get around issues with minification, we inject the dependencies as strings which will not be minified.

There are several issues with this approach as we're getting more data than we need in two senses:

  • We're getting all SubReddits, we could filter them but that would mean retrieving all the data from the server and then filtering them, which is as inefficient as it sounds. 
  • Furthermore, we're retrieving all properties of the SubReddit objects when we only need the name.
Let's deal with the first of these issues. We add a new method called GetDefaultSubReddits to the Repository, make sure you add it to the Interface as well:

        public IEnumerable<SubReddit> GetDefaultReddits()
        {
            return ctx.SubReddits.Where(x => x.IsDefault && x.IsPublic && !x.IsBanned);
        }
And we also add a call to this method from the Controller:
.        [HttpGet("[action]")]
        public IEnumerable<SubReddit> GetDefault()
        {
            return repository.GetDefaultSubReddits();
        }
Note that I'm using an action here, which is effectively the name of the method
Now we modify the redditApp.js
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', ['$scope', '$http', function($scope, $http) {
  $http.get("/api/SubReddit/GetDefault").then(function(data){
  $scope.subReddits =data.data ;
})}
])
  
})();
At this point we've solved issue number one but we're still sending quite a bit of unnecessary data up and down the wire so let's build a DTO that contains just the data that we need, which in this case is simply the Name. There are other ways of achieving this, for instance we could use an OData query to simply retrieve the necessary data.

This is the SubRedditDTO

namespace Reddit.Models
{
    public class SubRedditDTO
    {
        public string Name { get; set; }
    }
}
Modified GetDefault Method, I'm not sure whether it's a better idea to transform to DTO object here or on the Repository. I guess it depends on usage, in other words, if the Repository method will be used somewhere else it makes sense to do the transformation here.
        [HttpGet("[action]")]
        public IEnumerable<SubRedditDTO> GetDefault()
        {
            return repository.GetDefaultSubReddits().Select(x => new SubRedditDTO() { Name = x.Name });
        }
This is the reponse from the server now
[{"Name":"Gadgets"},{"Name":"Sports"},{"Name":"Gaming"},{"Name":"Pics"},{"Name":"WorldNews"},{"Name":"Videos"},{"Name":"AskReddit"},{"Name":"Aww"}]
This is before, when we where not using a DTO.
[{"SubRedditId":3,"Name":"Gadgets","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":4,"Name":"Sports","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":5,"Name":"Gaming","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":6,"Name":"Pics","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":7,"Name":"WorldNews","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":8,"Name":"Videos","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":9,"Name":"AskReddit","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true},{"SubRedditId":10,"Name":"Aww","IsPublic":true,"Posts":null,"Users":null,"IsBanned":false,"IsDefault":true}]
This might seem like a paltry gain but little savings like this can quickly add up.

Saturday 2 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 4

In the previous post we ended up again with a broken, although partially this time, application let's try to fix it.

The problem was that it could not resolve the service type for IRedditRepository as we were not telling how to inject the dependency properly, which is done on the ConfigureServices method on Startup.cs:

public void ConfigureServices(IServiceCollection services)
        {
            // Add framework services.
            services.AddMvc();
            services.AddEntityFramework()
                .AddSqlServer()
                .AddDbContext<RedditContext>();

            services.AddScoped<IRedditRepository, RedditRepository>();
        }
AddScoped will use the same instance for each request, could have used AddSingleton or AddInstance.

We try running this again and we see this:


So it's working but there is nothing there, we'll have a look at this later on.

Now let's try to look at the homepage, which at the moment looks like this.

This doesn't look like Reddit, so let's do a bit of editing.

Delete the Contact.cshtml and About.cshtml files, in the Home folder

Edit Index.cshtml so that it looks like this:

@{
    ViewData["Title"] = "Reddit";
}
This is the _Layout.cshtml file, which can be thought of as a master page or template page. I've removed pretty much everything from it that might be rendered.
<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>@ViewData["Title"] - Reddit</title>

    <environment names="Development">
        <link rel="stylesheet" href="~/lib/bootstrap/dist/css/bootstrap.css" />
        <link rel="stylesheet" href="~/css/site.css" />
    </environment>
    <environment names="Staging,Production">
        <link rel="stylesheet" href="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.5/css/bootstrap.min.css"
              asp-fallback-href="~/lib/bootstrap/dist/css/bootstrap.min.css"
              asp-fallback-test-class="sr-only" asp-fallback-test-property="position" asp-fallback-test-value="absolute" />
        <link rel="stylesheet" href="~/css/site.min.css" asp-append-version="true" />
    </environment>
</head>
<body>
    <div>
        @RenderBody()       
    </div>

    <environment names="Development">
        <script src="~/lib/jquery/dist/jquery.js"></script>
        <script src="~/lib/bootstrap/dist/js/bootstrap.js"></script>
        <script src="~/js/site.js" asp-append-version="true"></script>
    </environment>
    <environment names="Staging,Production">
        <script src="https://ajax.aspnetcdn.com/ajax/jquery/jquery-2.1.4.min.js"
                asp-fallback-src="~/lib/jquery/dist/jquery.min.js"
                asp-fallback-test="window.jQuery">
        </script>
        <script src="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.5/bootstrap.min.js"
                asp-fallback-src="~/lib/bootstrap/dist/js/bootstrap.min.js"
                asp-fallback-test="window.jQuery && window.jQuery.fn && window.jQuery.fn.modal">
        </script>
        <script src="~/js/site.min.js" asp-append-version="true"></script>
    </environment>

    @RenderSection("scripts", required: false)
</body>
</html>
The end result of all this should be an empty page.

I've left RenderBody and RenderSection method calls intact as they will come in handy later. In essence, they are used to merge in the content from the actual pages and to load the scripts needed respectively.

Since this is a modern app I'm going to use AngularJS, which is probably past it's prime now but there you go.

The simplest thing to do this is by using the manage bower packages:


This will display this page:


This is the result on the bower.json file, which should be noted is not shown by visual studio but it's there on the src\Reddit folder:

{
  "name": "ASP.NET",
  "private": true,
  "dependencies": {
    "bootstrap": "3.3.5",
    "jquery": "2.1.4",
    "jquery-validation": "1.14.0",
    "jquery-validation-unobtrusive": "3.2.4",
    "angular": "1.4.6"
  }
}
So let's add Angular to Reddit

I've added a new file called redditApp.js to wwwroot/js, which contains the following:
(function () {

  "use strict";

  var redditApp = angular.module('redditApp', []);

redditApp.controller('SubRedditListCtrl', function($scope) {
  $scope.subReddits = [
    {'name': 'Gadgets'},    
    {'name': 'Sports'},
    {'name': 'Gaming'},
    {'name': 'Pics'},
  ];
});
  
})();

I've hard coded a few subreddits to have something to display but we will soon get them from the database. In essence, the above is a self executing function containing an extremely simple angular controller that has an array of objects.

I have modified the Index.cshtml like this:

@{
    ViewData["Title"] = "Reddit";
}
@section scripts {

    <script src="~/lib/angular/angular.js"></script>
    <script src="~/js/redditapp.js"></script>
}
<div ng-app="redditApp">
    <div ng-controller="SubRedditListCtrl">
        <ul class="list-inline">
            <li ng-repeat="subReddit in subReddits">
                <span>{{subReddit.name}}</span>
            </li>
        </ul>
    </div>
</div>

Note the angular directives? ng-app, ng-controller and ng-repeat and also note the angular bingind {{subReddit.name}}

This is what Reddit looks like now:

Friday 1 January 2016

Cloning Reddit to learn ASP.NET 5 - Part 3

In the last post we applied a database migration but nothing occurred.

The reason for this was that the RedditContext class was not tracking any of the objects nor where the relationships specified on the OnModelCreating.

So I've modified the RedditContext class

using Microsoft.Data.Entity;

namespace Reddit.Models
{
    public class RedditContext : DbContext
    {
        public RedditContext()
        {
            //Database.EnsureCreated();
        }

        public DbSet<RedditUser> RedditUsers { get; set; }
        public DbSet<UserDetails> UserDetails { get; set; }
        public DbSet<SubReddit> SubReddits { get; set; }
        public DbSet<Post> Posts { get; set; }
        public DbSet<Comment> Comments { get; set; }
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            var connection = Startup.Configuration["Data:RedditContextConnection"];
            optionsBuilder.UseSqlServer(connection);
            base.OnConfiguring(optionsBuilder);
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<RedditUser>()
                .HasKey(x => x.RedditUserId);

            modelBuilder.Entity<UserDetails>()
                .HasKey(x => x.UserId);

            modelBuilder.Entity<RedditUser>()
                .HasOne(x => x.UserDetails)
                .WithOne(x => x.User);

            base.OnModelCreating(modelBuilder);
        }
    }
}

In order to recover from this mistake, we want to apply the new migration, but the Core migration contains nothing so I will first unapply it
dnx ef database update Start
dnx ef migrations remove
dnx ef migrations add Core
dnx ef database update
This time it worked .... hurray


At this point, it's worth considering how will this be deployed to a test environment and the answer is pretty simple. A migration script can be created with the following command:
dnx ef migrations script
This creates a Sql script that can be used to create the database.

It's time to have another look at our repository
Let's start with the IRedditRepository, where we now have a few methods and this time we'll try to implement some of them.
using System.Collections.Generic;

namespace Reddit.Models
{
    public interface IRedditRepository
    {
        UserDetails GetUserDetails();
        IEnumerable<SubReddit> GetSubscribedSubReddits(RedditUser user);
        IEnumerable<SubReddit> GetSubReddits();
        IEnumerable<Post> GetSubRedditPosts(SubReddit subreddit);
        IEnumerable<Comment> GetPostComments(Post post);
    }
}
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;

namespace Reddit.Models
{
    public class RedditRepository : IRedditRepository
    {
        private readonly RedditContext ctx;
        private readonly ILogger<RedditRepository> logger;

        public RedditRepository(RedditContext ctx, ILogger<RedditRepository> logger)
        {
            this.ctx = ctx;
            this.logger = logger;
        }

        public IEnumerable<Comment> GetPostComments(Post post)
        {
            return ctx.Posts
               .Where(x => x.PostId == post.PostId)
               .Single()
               .Comments;
        }

        public IEnumerable<Post> GetSubRedditPosts(SubReddit subreddit)
        {
            return ctx.SubReddits
                .Where(x => x.SubRedditId == subreddit.SubRedditId)
                .Single()
                .Posts;
        }

        public IEnumerable<SubReddit> GetSubReddits()
        {
            return ctx.SubReddits.AsEnumerable();
        }

        public IEnumerable<SubReddit> GetSubscribedSubReddits(RedditUser user)
        {
            throw new NotImplementedException();
        }

        public UserDetails GetUserDetails()
        {
            throw new NotImplementedException();
        }
    }
}

Straight away we see that there are going to be problems as we lack paging capabilities, but we'll add those later

In any case, we'll need to expose these method somehow, there are potentially two options here depending on what we're trying to accomplish: 

We could use these methods from inside a Controller or We could create an API and we're definitely going for the latter as it's way cooler

Create a new folder called Api inside the Controllers folder and add a new  Web API controller class, called SubRedditController
using System.Collections.Generic;
using Microsoft.AspNet.Mvc;
using Reddit.Models;

// For more information on enabling Web API for empty projects, visit http://go.microsoft.com/fwlink/?LinkID=397860

namespace Reddit.Controllers.Api
{
    [Route("api/[controller]")]
    public class SubRedditController : Controller
    {
        private readonly IRedditRepository repository;

        public SubRedditController(IRedditRepository repository)
        {
            this.repository = repository;
        }

        // GET: api/values
        [HttpGet]
        public IEnumerable<SubReddit> Get()
        {
            return repository.GetSubReddits();
        }

    }
}

We can now build and debug the project. This allow us to navigate to http://localhost:<portnumber>/api/SubReddit

Unfortunately this does not work:


We'll look at the solution to this problem in the next post.