James Gregory

Rails Production 500 Error Page Showing in Dev

We’re sorry, but something went wrong!

I’ve been seeing some strange behaviour on the development environment for On the Game recently. I’ve just spent an hour combing through every line of code that varies from a standard Rails app, and I’ve finally found the issue. I’m putting it here for posterity.

Regardless of my configuration, Rails was always displaying the production page for 500 errors (public/500.html) even in development mode.

I poked around for a while and eventually found it to be this:

I’d redefined Enumerable#sum somewhere (don’t ask!), and my implementation didn’t handle arrays of strings. That’s it, that’s why the detailed error page wasn’t being shown in development.

Ends up, Enumerable#sum is used by the development error page, and when something on that error page itself raises an error Rails will just fall back and show the 500.html and not log anything.

Moral of the story: If you’re seeing this behaviour, check you haven’t (badly) monkey patched something that the error page is using.

Learning. It Ain’t Hard.

“I don’t have enough free time to learn a new language!”, or tool, or framework, etc. I hear this often. I appreciate we all handle things differently, but here’s my take on it.

I remember years ago, some friends and I were talking about money and our savings accounts. They were flabbergasted at how I, a 19 year old at the time, were able to have nearly £10,000 saved (sadly which I’ve since spent!). I explained that it’s easy to save money if you do it slowly and consistently. £50 a week over the course of three years adds up. Obvious, right?

You don’t sit down and decide “I’m going to save £10,000” and then promptly spend one month working yourself to death to put away the whole amount in one go. It’d be nice to have the option to do that occasionally, but that’s not how most of us work. The same applies to learning.

You don’t need to read a 300 page book to learn something. Read a blog post (or an article, mailing list thread, commit discussion). One. Pick a subject and read a single article on it. Do this once a week, every week. That’s it, that’s my secret to learning.

Knowledge isn’t a substitute for experience, but it’ll certainly get you a lot further down the road, faster, than you would get without it.

Take Ruby as the typical example. Ruby on Rails has been around for 6 years now (and Ruby for nearly 15, but not visibly), if you’d have read one article on Ruby a week, you’d have read 312 by now. Three hundred and twelve. Combine that with the occasional Sunday afternoon playing around and I’m confident you’d be pretty proficient by now.

  • Javascript is around 15 years old.
  • Haskell, 20 years.
  • Erlang, 25 years.
  • Python, 20 years.
  • Clojure, 4 years. Scheme? 36 years.

Of course, we’ve not all been trying to learn those languages for as long as they’ve existed. Consider this though, in one year you could be proficient in a new language if you commit to just 15 minutes a week, or you could be in exactly the same place you are now still unable to find time to learn anything.

How do you eat an elephant? One bite at a time.

Addendum: but everything moves so fast!

No, it doesn’t. It just feels like that from the inside.

Learn something fundamental and your knowledge will transcend trends. Learning a new language is a great way to do that whilst also feeling cool and trendy. Ruby or Python will help you understand dynamic languages, Scala will make you appreciate how powerful and unobtrusive static languages can be, and Clojure will introduce the code-as-data mindset. All those things will be useful beyond the scope of the language you’re learning.

Stealth edit: Fixed my amazing saving ability.

Rhino Licensing

As one of the last things on my to-do list for Inca, licensing has been delayed and delayed; finally, with me making the announcement on Twitter the other day I felt I should probably get it sorted. I was going to write a nice long post about my thoughts on anti-piracy measures, but the Balsamiq team have already done it nicely.

Instead, here’s a guide to using Rhino Licensing. Your mileage may vary.

Rhino Licensing is an open-source licensing framework by Ayende Rahien, and it grew out of his frustration with other license providers while creating NHibernate Profiler.

Rhino Licensing uses asymmetric encryption–also known as public-key cryptography–specifically the RSA algorithm. In cryptography, asymmetric means there are two parts to each key, one public and one private. You encrypt a value using the one key, and it can only be decrypted using the other key. In the case of license key generation, we store our private key on the server (and never tell anyone it!), and distribute the public key with our application. When the user receives a license key, the application is able to verify that it came from us by using the public key. If someone tries to use a license key they’ve generated themselves, it wouldn’t work unless they had our exact private key.

Other systems that use asymmetric encryption include SSH, SSL, PGP, and among others, Git uses asymmetric encryption for its security due to it using SSH.

The format of the license that Rhino Licensing generates is an XML document, plain and simple. By default, it includes the user’s name, an expiration date, the type of license (trial, standard, floating, etc…), and most importantly the cryptographic signature. Below is an example Rhino Licensing generated license.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<?xml version="1.0" encoding="utf-8"?>
<license id="ae4c05b5-c188-47f8-852f-b4e5375621f7"
    expiration="2011-01-01T00:00:00.0000000" type="Standard">
  <name>Bilbo</name>
  <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
    <SignedInfo>
      <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315" />
      <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />

      <Reference URI="">
        <Transforms>
          <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
        </Transforms>
        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
        <DigestValue>sKDT7gzgedzmh1AMxxLbfcF1Hsw=</DigestValue>
      </Reference>
    </SignedInfo>
    <SignatureValue>
        SfkvCGv1+EdLTvaROv27ymDumS0y02fPANTVhr0Yxd/
        AxxVH0q0BQ6w8Ou5L7gyLYLvnSckgjhrGnGpiifdvbg==
    </SignatureValue>
  </Signature>
</license>

What’s a signature?

A cryptographic signature is a string of bytes used to aid in verification of the source of a document. In most cases–and in the case of Rhino Licensing–the signature is generated by hashing the content of the file using SHA1 (or another suitable hashing algorithm), and encrypting that hash using the private key. The resulting string of bytes is the signature.

When the client receives a file with a signature, it creates a hash of the file (sans-signature) and compares that hash to the decrypted value of the signature. Baring in mind that the signature is an encrypted hash of the file, if either of the public or private keys were invalid or the file had been tampered with, then the decrypted string would not match the hash of the file. When the client generated hash and the decrypted signature match, you can be confident the sender of the message is who you think it is.

The great part about using the hash of the document as the signature is that if the user tampers with the license–such as changing the expiration date or the licensee name–the signature will no longer match the hash the client generates, and BAM, instant invalid license.

One particular aspect I like with this approach is that the licensee name is clearly visible in the license. This simple act can be quite dissuasive when it comes to piracy; after all, who would want their name visible on all the torrent websites out there?

Enough with the waffle, let’s start using Rhino Licensing.

Create your key

The first thing we need to do is generate a public/private key pair for us to use in our license generation. Rhino Licensing uses the RSACryptoServiceProvider for it’s keys, so we can use that ourselves to generate our keys. One of the simplest ways to do that is to knock up a really quick console application.

1
2
3
4
5
6
7
8
9
10
11
12
13
class Program
{
  static void Main(string[] args)
  {
    var rsa = new RSACryptoServiceProvider(1024);

    File.WriteAllText("publicKey.xml", rsa.ToXmlString(false));
    File.WriteAllText("privateKey.xml", rsa.ToXmlString(true));

    Console.WriteLine("Done");
    Console.ReadKey();
  }
}

That program will generate a new key for you and write out the public and private parts to two separate XML files (publicKey.xml and privateKey.xml). Keep these two files handy!

You will probably want to change the 1024 value in the RSACryptoServiceProvider constructor to something larger. This is the size of the key that will be generated. The larger the better really, but larger numbers require longer to generate.

Update your application

Now we’ll update your application to complain if it doesn’t have a license. In your entry-point for your application you need to assert the license. Before we do that, make sure you’ve got your public key handy from the previous step. Rhino Licensing needs the content of this file, so you can either embed it directly into a constant or as a resource. Secondly, it needs to know where to look for a license that the user will have been given.

Assuming a really simple app, here’s a Program.cs with Rhino Licensing:

1
2
3
4
5
6
7
8
9
10
11
12
13
class Program
{
  static void Main()
  {
    var publicKey = File.ReadAllText("publicKey.xml");

    new LicenseValidator(publicKey, "license.xml")
      .AssertValidLicense();

    Console.WriteLine("Hello");
    Console.ReadKey();
  }
}

If the license.xml file doesn’t exist, or if it’s invalid, Rhino Licensing will throw an exception. Of course, as we haven’t generated a license yet we’ll get an exception. That’s the client done though, so we can move on to our final step now. Once we complete that step, you will be able to just drop a license.xml into the same directory as your application and everything will work as expected.

Create the license generator

We need something to actually generate a license for your users. If you were being all fancy you could do this as a web-service that gets integrated with your payment provider; however, for simplicity’s sake we’ll just create another console application that spits out a license.

Rhino Licensing has a few prerequisites for a license: A name, an id of the license, an expiration date, and a license type. You can also supply a dictionary of custom values, but we wont be using that in this example.

For a really simple license generator all you need is the above for each license, and the private key. We’ll wrap that up in a simple console app which’ll prompt for the required info:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Program
{
  static void Main(string[] args)
  {
    var privateKey = File.ReadAllText("privateKey.xml");

    var id = Guid.NewGuid();
    var generator = new LicenseGenerator(private_key);

    Console.Write("Name: ");
    var name = Console.ReadLine();

    Console.Write("Date: ");
    var expirationDate = DateTime.Parse(Console.ReadLine());

    Console.Write("Type: ");
    LicenseType licenseType = (LicenseType)Enum.Parse(typeof(LicenseType), Console.ReadLine());

    // generate the license
    var license = generator.Generate(name, id, expirationDate, licenseType);

    Console.WriteLine();
    Console.WriteLine(license);
    Console.ReadKey();
  }
}

That’s it, you’ve got a license generator. Remember, this will exist behind your firewall, the users will (and must) never see your private key.

If you take the output of this application and save it in a license.xml file in your client, your application should stop throwing license exceptions and run normally! You’ve officially just licensed your app.

What have we just done?

  • Created a private key generator, purely for convenience than anything else. We should only need to run this once before we start generating our keys. You could run it once per major release to ensure your 1.0 licenses are never compatible with 2.0.
  • Updated our client application to validate the presence and validity of a license (and to explode if there isn’t one).
  • Created a “server” to generate licenses for us. This is very primitive but could easily be adapted to a web server. You might want to look at the Rhino Licensing LicensingService before you continue.

NDC 2010

NDC 2010 was a huge success, if you ask me. This was largely down to the NDC team, who deserve all the praise they’re getting (and much more). Unlike conferences I’ve been to in the past, NDC was truely by the people, for the people. Scott Bellware put it much better than I could, with his praise for Norwegian Developers (and Kjersti Sandberg). Herself and the rest of the team were there because they wanted to be, not because they had to. The whole attitude surrounding this conference was one of learning, not plugging products or motives. I could go into great detail about the individual aspects of the conference, but it’s much easier to say that there’s nothing I’d change. If there’s one conference you should go to next year, make it NDC 2011.

My part in all this…

I presented on two topics at NDC; the first was an introduction to Fluent NHibernate, and the second an introduction and demo of Git. Fluent NHibernate was on the Thursday, and Git the Friday. It was a great experience speaking at an event of this size, and the number of people who seemed genuinely interested in what I had to say (by either turning up to my talks, or approaching me afterwards) was quite humbling. If you’re interested in seeing what I presented, the talks will be on the NDC website before too long.
The videos are becoming available at streaming.ndc2010.no, but they’re suffering from bandwidth issues. If you do want to watch the videos, please stream them until the issues are resolved.
Due to me speaking, and also being woefully unprepared, I didn’t get to see as many talks as I would’ve liked — regardless of how well rehearsed and prepared my talks are, it seems I always end up rewriting them an hour before I go on — I did end up catching Lisa Crispin, Steve Strong, Eric Evans, Greg Young, Jon Skeet, Roy Osherove, Michael Feathers, Sebastian Lambla, and Mark Nijhof. I recommend you catch their talks online when they’re published. The quality of the talks at NDC have been significantly higher than I’ve seen elsewhere, and (more importantly) the technical level seems to be higher too; there was very few basic talks (a few introductions, mine included, but no pandering), but the majority were pretty in-depth. This is quite a contrast to the all-too-often “Microsoft technology of the week” presentations you can get at conferences. Of course, a big part of any conference is what happens between (and after) the talks. Meeting other developers is a big part of why I go to these things, after all I don’t get out much normally. The socialising aspect of NDC was excellent, good drinks and food was had by all (I have a good idea they did anyway, at least one place we ate refused to serve us as a group anything other than the same meal for everyone!). I can’t possibly list everyone who I met while I was out there, but it was really good to meet up with some people who I’ve known on Twitter for a long time; specifically Hadi Hariri, Steve Strong, Greg Young, Ben Hall, Seb Lambla, Mark Nijhof, and Rob Conery are ones that stand out to me right now. NDC 2011 will be next year, and it’s looking like it’ll be in May this time. If there’s anyone that’s on the fence about going, I would whole heartedly recommend you do. Even more so, I’d recommend you submit a talk of your own. It’s a great way to test yourself, if nothing else. I’ll definitely be going next year; hopefully I’ll speak again, but if not I’ll still be there. Hope to see you there!

Speaking Engagements in 2010

In an attempt to get out of my shell more this year, I’ve taken up speaking (at conferences, not just in general). First came my Git E-VAN, then came Pablo’s Fiesta, so what else have I got lined up this year? 24th April: Irish Open Spaces Coding Day II in Dublin, where I’ll be speaking about Fluent NHibernate. 16th-18th June: NDC 2010 in Oslo, speaking about Fluent NHibernate and DVCS with Git. The abstracts for my talks can be found below. If you’re interested in hearing me speak, or know of any opportunities that you think I might be interested in, please drop me an email (james@jagregory.com). You can see my availability on my speaking calendar.

Fluent NHibernate

An introduction and overview to object⁄relational mapping using Fluent NHibernate. See how Fluent NHibernate can help you map your domain with the least amount of effort, how you can remain flexible with your database, and how to drive your design through convention–over–configuration; all without writing a single line of XML. The talk is an introduction to Fluent NHibernate for those that aren’t familiar with it, and assumes some NHibernate experience and is for .Net developers primarily. The goal is to show people how low–impact NHibernate can be with Fluent NHibernate, and how it can actually speed up development in rapid–change environments. I’ll cover an overview of what Fluent NHibernate is, the various parts of it (the fluent interface, the conventions, and the auto–mappings, and the configuration aspect), then expand on how these features can be utilised to improve your NHibernate experience and simplify your development process.

Introduction to Git

An introduction to distributed version control using Git, and how your VCS should work with you and not against you. How DVCS can completely alter your development process, streamline it, and help you produce better software, faster. Covering how local repositories speed up your development, multiple authoritative sources, distributed teams, multiple workflows, and some of the more distinct features of Git. With experiences from an OSS team on how the migration from SVN to Git has helped the project and changed how the team works (Fluent NHibernate).

Git E-VAN Recording

Last monday (8th of Feb) I did an E-VAN on Git; an introductory talk on Git and DVCS, covering pretty much everything you need to know for day-to-day Git life. I think it went down well, certainly didn’t hear anyone complaining.

The talk was recorded, so if you haven’t already seen it then you can do so at your own leisure.

The video is up on the E-VAN’s Vimeo account, specifically here.

Git E-VAN

Just a reminder that tonight I’ll be doing an E-VAN on Git tonight at 7pm GMT.

It’s going to be a pretty basic talk on what Git is (and indirectly what DVCS is), and a demo on how to use most of the features you’ll need daily. There might be some advanced talk at the end, depending on how well I time things.

If you’ve already used Git (successfully), then this talk won’t really be for you; unless you just want to listen to me ramble for an hour and an half.

Looking forward to “seeing” you there. Now, I best finish reading that “Getting started with Git” book I’ve had lying around for months.

Git: Remotes, Contributions, and the Letter N

Here’s a few ways to think about Git and it’s distributed nature.

  • You deal with multiples of repositories, not a single central repository
  • Updates come from a remote repository, and changes are pushed to a remote; none of these repositories have to be the same
  • Origin is the canonical name for the repository you cloned from
  • Upstream is the canonical name for the original project repository you forked from

General pushing and pulling

Figure 1

Pushing your changes to a remote: git push remote_name

Figure 2

Pulling changes from a remote: git pull remote_name

Or if you want to rebase:

git fetch remote_name
git rebase remote_name/branch
You can change your branch.autosetuprebase to always, to make this the default git pull behaviour.

That’s all there is to moving commits around in Git repositories. Any other operations you perform are all combinations of the above.

Github — personal repositories

When you’re dealing directly with Github, on a personal project or as the project owner, your repositories will look like this:

Figure 3

To push and pull changes between your local and your github repositories, just issue the push and pull commands with the origin remote:

git push origin
git pull origin

You can set the defaults for these commands too, so the origin isn’t even necessary in a lot of cases.

Github — receiving contributions

As a project owner, you’ll sometimes have to deal with contributions from other people. Each contributor will have their own github repository, and they’ll issue you with a pull request.

Figure 4

There’s no direct link to push between these two repositories; they’re unmanned. To manage changes from contributors, you need to involve your local repository.

You can think of this as taking the shape of a V.

Figure 5

You need to register their github repository as a remote on your local, pull in their changes, merge them, and push them up to your github. This can be done as follows:

1
2
3
git remote add contributor contributor_repository.git
git pull contributor branch
git push

Github — providing contributions

Do exactly as you would your own personal project. Local changes, pushed up to your github fork; then issue a pull request. That’s all there is to it.

Github — the big picture

Here’s how to imagine the whole process, think of it as an N shape.

Figure 6

On the left is the contributor, and the right is the project. Flow goes from bottom left, along the lines to the top right.

  1. Contributor makes a commit in their local repository
  2. Contributor pushes that commit to their github
  3. Contributor issues a pull request to the project
  4. Project lead pulls the contributor’s change into their local repository
  5. Project lead pushes the change up to the project github

That’s as complicated as it gets.

Behaviours in MSpec

Cross posted to Los Techies

MSpec is awesome, I think it’s praised by myself and others enough for that particular point to not need any expansion; however, there is a particular feature I would like to highlight that hasn’t really got a lot of press: behaviours.

Behaviours define reusable specs that encapsulate a particular set of, you guessed it, behaviours; you’re then able to include these specs in any context that exhibits a particular behaviour.

Lets go with the cliche’d Vehicle example. Given an IVehicle interface, with Car and Motorbike implementations; these all expose a StartEngine method and some properties reflecting the state of the vehicle. We’ll start the engine and verify that it is actually started, whether it’s got anything on the rev counter, and whether it’s killing our planet in the process (zing!).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public interface IVehicle
{
  void StartEngine();
  bool IsEngineRunning { get; }
  bool IsPolluting { get; }
  int RevCount { get; }
}

public class Car : IVehicle
{
  public bool IsEngineRunning { get; private set; }

  public void StartEngine()
  {
    // use your imagination...
  }
}

public class Motorbike : IVehicle
{
  public bool IsEngineRunning { get; private set; }

  public void StartEngine()
  {
    // use your imagination...
  }
}

Those are our classes, if rather contrived, but they’ll do. Now what we need to do is write some specs for them.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
public class when_a_car_is_started
{
  Establish context = () =>
    vehicle = new Car();

  Because of = () =>
    vehicle.StartEngine();

  It should_have_a_running_engine = () =>
    vehicle.IsEngineRunning.ShouldBeTrue();

  It should_be_polluting_the_atmosphere = () =>
    vehicle.IsPolluting.ShouldBeTrue();

  It should_be_idling = () =>
    vehicle.RevCount.ShouldBeBetween(0, 1000);

  static Car vehicle;
}

public class when_a_motorbike_is_started
{
  Establish context = () =>
    vehicle = new Motorbike();

  Because of = () =>
    vehicle.StartEngine();

  It should_have_a_running_engine = () =>
    vehicle.IsEngineRunning.ShouldBeTrue();

  It should_have_a_running_engine = () =>
    vehicle.IsEngineRunning.ShouldBeTrue();

  It should_be_polluting_the_atmosphere = () =>
    vehicle.IsPolluting.ShouldBeTrue();

  It should_be_idling = () =>
    vehicle.RevCount.ShouldBeBetween(0, 1000);

  static Motorbike vehicle;
}

Those are our specs, there’s not much in there but already you can see that we’ve got duplication. Our two contexts contain identical specs, they’re the same in what they’re verifying, the only difference is the vehicle instance. This is where behaviours can come in handy.

With behaviours we can extract the specs and make them reusable. Lets do that.

Create a class for your behaviour and adorn it with the Behaviors attribute — this ensures MSpec recognises your class as a behaviour definition and not just any old class — then move your specs into it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[Behaviors]
public class VehicleThatHasBeenStartedBehaviors
{
  protected static IVehicle vehicle;

  It should_have_a_running_engine = () =>
    vehicle.IsEngineRunning.ShouldBeTrue();

  It should_have_a_running_engine = () =>
    vehicle.IsEngineRunning.ShouldBeTrue();

  It should_be_polluting_the_atmosphere = () =>
    vehicle.IsPolluting.ShouldBeTrue();

  It should_be_idling = () =>
    vehicle.RevCount.ShouldBeBetween(0, 1000);
}

We’ve now got our specs in the behaviour, and have defined a field for the vehicle instance itself (it won’t compile otherwise). This is our behaviour complete, it defines a set of specifications that verify that a particular behaviour.

Lets hook that behaviour into our contexts from before:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public class when_a_car_is_started
{
  Establish context = () =>
    vehicle = new Car();

  Because of = () =>
    vehicle.StartEngine();

  Behaves_like<VehicleThatHasBeenStartedBehaviors> a_started_vehicle;

  protected static Car vehicle;
}

public class when_a_motorbike_is_started
{
  Establish context = () =>
    vehicle = new Motorbike();

  Because of = () =>
    vehicle.StartEngine();

  Behaves_like<VehicleThatHasBeenStartedBehaviors> a_started_vehicle;

  protected static Motorbike vehicle;
}

We’ve now put to use the Behaves_like feature, which references our behaviour class and imports the specs into the current context. Now when you run your specs, the specs from our behaviour are imported and run in each context. We don’t need to assign anything to that field, just defining it is enough; the name you choose for the field is what’s used by MSpec as the description for what your class is behaving like. In our case this is translated roughly to “when a motorbike is started it behaves like a started vehicle”.

There are a couple of things you need to be aware of to get this to work: your fields must be protected, both in the behaviour and the contexts; and the fields must have identical names. If you don’t get these two correct your behaviour won’t be hooked up properly. It’s also good to know that the fields do not need to have the same type, as long as the value from your context is assignable to the field in the behaviour then you’re good; this is key to defining reusable specs for classes that share a sub-set of functionality.

In short, behaviours are an excellent way of creating reusable specs for shared functionality, without the need to create complex inheritance structures. It’s not a feature you should use lightly, as it can greatly reduce the readability of your specs, but it is a good feature if you’ve got a budding spec explosion nightmare on your hands.

Git’s Guts: Merging and Rebasing

Cross posted to Los Techies

Here we go again, explaining the internals of Git with the intention of helping you understand what you’re doing day-to-day. Last time I covered branches, HEAD, and fast-forwarding. Today we’ll dive into the guts of merging and rebasing.

Merging branches

You’ve probably merged before. You do it when you want the changes from one branch in another. The principal is the same in Git as it is most other source control systems, but the implementation differs.

Given the following commit structure, consisting of two branches created from the same commit, each with two commits after the branching occurred.

Figure 1

When these two branches are merged together, this is the structure that results:

Figure 1

The top-most commit, the red one, is a new commit made by the merge; the merge commit is what reminds Git that a merge occurred next time it’s showing the history. This commit is special, as it contains multiple parent’s in it’s meta-data; these multiple parent’s allow Git to follow the two trees of commits that constituted the branches that were merged.

One difference in how Git handles merges compared to many other SCMs is that it preserves the commits that were made in both branches. In other systems merges are often represented as a single commit containing the squashed contents of all the commits that were made in the branch being merged in. Git doesn’t do this (by default, you can tell it to if you want), and therefore preserves all the commits just as they were made; this is quite nice, as it proves incredibly useful to be able to track the origin of changes beyond the point of a merge.

When you merge two branches, it’s interesting to know that none of the commits are altered in the process. Just bare this in mind for now, I’ll explain why this is good to know later.

After a merge, if you were to view the history, you’d see it shown like the previous example, commits in chronological order; the feature branch commits are interspersed between the master commits.

Figure 1

Yet no commits have been altered in the merge, so how are the commits in a different order? Well, they’re not, Git’s just showing you it in the order you expect it to be in. Internally the structure is still as below:

Figure 1

The merge commit instructs Git to walk the two trees while building the history, and it just displays the results in chronological order. This makes more sense if you recall that Git commits don’t hold differences like other SCM systems, instead they each contain a snapshot of the complete repository; while in another SCM the ordering of commits is vital — otherwise the diffs wouldn’t build a valid file — Git is able to infer order without affecting the repository contents.

Looking at it in commit order, you can quite easily see how Git flattens the history to be perceived as linear without ever having to touch any of the original commits.

What happens if there’s a merge conflict?

We’ve all dealt with conflicts in merging before. They typically happen when changes are made to the same file in two branches, in a way that cannot be easily merged (two people edit the same line, for example).

Git’s commit’s are immutable though, so how are the changes that you need to make to resolve these conflicts saved? Simple. The merge commit is a regular commit with some extra meta-data, and so it capable of containing changes itself; merge conflict changes are stored in the merge commit. Again, no changes necessary to the original commits.

Git objects, immutability, and rewriting history

A Git repository is comprised of objects. A file is a blob object with a name attached to it; if you have two files with the same content, that’s just two names to a single blob. A directory is a tree object, which is comprised of other trees and blobs. A commit is an object that references a tree object, which is the state of the repository at the time of committing.

To read more about git objects, I’d definitely recommend you read the Git community book.

Git objects are immutable. To change an object after it’s been created is impossible, you have to recreate the object with any changes made. Even operations that seem to modify objects actually don’t; commit --amend is a typical example, that deletes and re-creates the commit rather than actually amending it.

I mentioned that merges don’t rewrite history, and that it’s a good thing. Now I’ll explain why. When you rewrite history, you do so by making changes to commits that ripple up the commit tree; when this happens, it can cause complications when others merge from you. Given a series of commits, like so:

Figure 1

You then share these commits with another user.

Figure 1

John now has Michael’s commits in his repository; however, Michael notices he’s made a typo in the first commit message, so he amends the commit message. The change in the message requires the commit be recreated. With that first commit recreated, the second commit now has an invalid parent reference, so that commit has to be recreated with the new reference; this recreation ripples it’s way up the tree, recreating each commit with a new parent. Michael has completely rewritten his tree’s history.

Figure 1

Notice all the commit hashes have changed in Michael’s repository, and John’s now don’t match. If Michael was then to make a new commit to his repository, and John tried to merge that change into his repository, Git would get very upset because the new commit would reference a commit that doesn’t exist in John’s repository.

The golden rule is: rewriting history is fine as long as the commits that will be affected haven’t been made public.

Rebasing

The purpose of a rebase is the same as a merge, to bring two tree’s of commits together. It differs in it’s approach. Rebasing is a seriously sharp tool. Very powerful, but pretty easy to cut yourself with it.

When you rebase one branch onto another, Git undoes any changes you’ve made in the target branch, brings it up to date with the changes made in the source branch, then replays your commits on top. This sounds quite strange, so I’ll go over it step-by-step.

You start with your diverged branches:

Figure 1

If you then rebase feature onto master, Git undoes the changes in master.

Figure 1

The history of both branches is now the same, master has been updated to reflect feature; the new commits that were made in master are now detached, floating in the repository without anything referencing them.

The next step is to replay the master commits onto the new structure. This is done one-by-one, and can sometimes result in conflicts that will need to be handled like any merge.

After replaying the repository will look like this:

Figure 1

The master branch commits are now on the top of the stack, after the commits from the feature branch.

You should recall that commits are immutable, and for changes to be made commits need to be recreated. A rebase is a destructive operation, as it has to rewrite commits to be able to work. In this case, the commits from feature have been unaffected, but the master commits have been assigned new parents (and thus rewritten). What’s also noticeable is there’s a lack of a merge commit, which isn’t needed because the commits have been integrated into the tree; any conflicts are stored in the amended commits, rather than in a merge commit.

The rewriting of commits in a rebase is what makes it a dangerous operation to perform on any branch that has already been pushed to the public (or specifically, that the changes affected by the rebase have already been pushed to the public). A rebase can cause problems upstream, like mentioned in the previous section.

Rebase has it’s place though. If you’re working locally and haven’t yet pushed your changes public, it can be a useful tool. Rebase can be used to pull in changes from upstream in the order that the upstream repository has them, and your local changes (that can be rewritten because you’re the only one with them) can be replayed on-top; this is a really easy way to keep your repository up-to-date with an authoritative source. You can also use Rebase to manage local branches that you don’t necessarily want polluting the history with merge markers.

When to rebase and when to merge?

Merge when you’ve already made changes public, and when you want to indicate that two tree’s have converged. Rebase pretty much any other time.

That’s it for this time. Same deal as last time, if you have anything you’d like me to cover I’ll nail it in the next one.