The new version of git - 1.8.2 is here and with it, git is better at ignoring things. The files and directories in your repository, that is.

From the 1.8.2 release notes you can see the following:

The patterns in .gitignore and .gitattributes files can have **/, as a pattern that matches 0 or more levels of subdirectory. E.g. “foo/**/bar” matches “bar” in “foo” itself or in a subdirectory of “foo”.

This is great for ignoring, say lib folders within your src folder, but not ignoring lib folders elsewhere, especially, the root of your repsoitory. All the gross ways of achieving something like the above can be, well, ignore, and we can start using this.

Wait! 1.8.2 has even more in the art of ignoring thing:

“git check-ignore” command to help debugging .gitignore files has been added.

This seems neat! I have seen so many people struggle with trying to understand what is being ignored and what is not. Let’s see how this new command can help.

You can pass path(s) to the git check-ignore command and it will tell you the pattern from .gitignore or elsewhere that ends up ignoring the path(s).

Let’s try it out:

1
2
3
4
5

git init .
touch afile
mkdir bin
touch bin/abin.dll
echo "bin/" > .gitignore

Now if you do git check-ignore bin or git check-ignore bin/abin.dll, the command will spew back the paths, confirming that they are ignored. Of course, the git check-ignore command can go even further:

1
2

$ git check-ignore bin/a.dll --verbose
.gitignore:1:bin/   bin/a.dll

When you use the --verbose flag, it tells you the source of the ignore pattern, the line number, the pattern itself and the path being ignored because of the pattern.

While most of the times our git ignores are pretty simple like bi/ or *.dll, but for the times when things go out of hand, it makes sense to use the git check-ignore command to debug what is going on.

The 1.8.2 version of Git is here and one of the most sought features in git finally reaches the fans - the pre-push hook.

Go ahead, install the new version of git and do a git init or git init --bare. Under the hooks folder, you should see a sample for the brand new pre-push hook. Yay!

This is what the top comments of the pre-push script say:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

#!/bin/sh

# An example hook script to verify what is about to be pushed.  Called by "git
# push" after it has checked the remote status, but before anything has been
# pushed.  If this script exits with a non-zero status nothing will be pushed.
#
# This hook is called with the following parameters:
#
# $1 -- Name of the remote to which the push is being done
# $2 -- URL to which the push is being done
#
# If pushing without using a named remote those arguments will be equal.
#
# Information about the commits which are being pushed is supplied as lines to
# the standard input in the form:
#
#   <local ref> <local sha1> <remote ref> <remote sha1>
#
# This sample shows how to prevent push of commits where the log message starts
# with "WIP" (work in progress).

That seems pretty simple, and in line with other hooks. The example is also pretty useful, helping you block pushes containing commits with message starting with “WIP”.

So what are the scenarios for using this? Come on, do you really need a reason to use something as good as this? Well anyways, one thing that comes into mind is automatically running your local builds / tests just before pushing the commits to remote. If they fail, don’t push.

One of the first things that people coming from the Ruby / Rails world and entering the amazing world of Python / Django ( ahem ) want to know is what is the equivalent of rvm? Or to be more generic, how does one manage their “pythons” and the manage their “gemsets”

I see that newcomers to Python are still being pointed to virtualenv and then to the virtualenvwrapper which apparently makes working with virtualenv much easier. As someone coming back to the world of Python, from the days of installing all the packages for every project that your are working on system wide, I too, was led towards the aforementioned duo.

Until I came across pythonbrew, that is. It describes itself rather modestly as “..a program to automate the building and installation of Python in the users $HOME”, and claims to be inspired by rvm itself.

What many don’t seem to realize is that pythonbrew itself is a wrapper to virtualenv and allows the ability to manage virtual environments in addition to your pythons. The interface is consistent and it does more than what virtualenv or virtualenvwrapper do.

This is how you create and use virtual environments with pythonbrew:

1
2

pythonbrew venv create proj
pythonbrew venv use proj

This is how you do it with virtualenv:

1
2

virtualenv proj
source bin/activate

Bit better with virtualenvwrapper:

1
2
3

mkvirtualenv proj #this also activates it
#later
workon proj

Can you see the consistency in interface with pythonbrew that I am talking about? One you start with pythonbrew, you know where to go next. What the next step would be. Maybe I am spoilt by rvm, but pythonbrew just seems to be right approach here for me. The fact that it uses virtualenv under the wraps means you can always go down a level and use it directly.

With the 2.7.x python and 3.x python being used simultaneously, I surely hope people do need easy ways to install and use multiple pythons, in addition to managing the packages.

I have been using pythonbrew in my Django project and have had absolutely no problems with it. I had no problems setting up Pycharm with it, as well. It is exactly what I needed, without the complexity of the virtualenv interface, with a consistent way of doing things, and similarity to rvm.

So my question to pythonistas is, why do you still use / recommend virtualenv ( alone ) when there is pythonbrew? Or am is just reading old docs and blogs?

This is a follow up to the previous post on things that trip newbies in Powershell

In this post, I will be concentrating on Encodings. When I began using Powershell, it managed to confuse, trick and irritate me a lot. Encodings issues continue to be problematic with Powershell even in v3.0.

Let me start off with a common enough usage that might come to bite you. Say you have a git repo. You can want to pipe some content to .gitignore to ignore some files and directories. Easy right?

You can do:

1

"*.dll" > .gitignore

As any git user would know, that should ignore any dll file in your repo. But will it? Go ahead try it out.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

PS> git init .
PS> touch a.dll
PS> "*.dll" > .gitignore
PS> git add .
PS> git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#       new file:   .gitignore
#       new file:   a.dll

What??? Shouldn’t a.dll be ignored. Let’s see what’s going on here:

1
2
3
4
5
6
7
8

PS> git diff --cached
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..2f14638
Binary files /dev/null and b/.gitignore differ
diff --git a/a.dll b/a.dll
new file mode 100644
index 0000000..e69de29

So git is treating the .gitignore file as a binary. This is where encodings come into the picture.

Doing "*.dll" > .gitignore is pretty much the same as doing "*.dll" | out-file .gitignore. The default encoding that is used here is UCS-2. Many tools like git are not happy with it, at least by default.

We have this, from the docs:

By default, the Out-File cmdlet creates a Unicode file. This is the best default in the long run, but it means that tools that expect ASCII files will not work correctly with the default output format. You can change the default output format to ASCII by using the Encoding parameter

To make git happy, you can convert the .gitignore file to, say ascii. Actually, the out-file cmdlet provides ability to do just that.

1

 "*.dll" | out-file -Encoding ascii .gitignore

Now, the dlls should be ignored as expected. Unfortunately, there is no way to change how > behaves in this aspect!

The most easiest way to get AppSettings in .NET is to use the ConfigurationManager. It goes like:

1

var debugSetting = ConfigurationManager.AppSettings["IsDebug"];

The AppSettings property is just a NameValueCollection - an association of string keys and string values. You pass in the string key whose value you want and you get the string value back.

So, if your App.config were like:

1
2
3
4
5
6

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="IsDebug" value="true" />
  </appSettings>
</configuration>

The debugSetting will have the string value “true”. But most often, you don’t want just the string. Even in this example, you want a boolean value to see if debug is enabled or not (contrived, yes). So, you end up doing something like:

1
2
3
4

var debugSetting = Convert.ToBoolean(ConfigurationManager.AppSettings["IsDebug"]);
if(debugSetting){
    //do something nasty
}

Same goes for integers, urls, and so many other types. Get back the string, convert it to the type that is needed, and consume it.

Is there a better way to all this, which is also pretty simple? TypeConverters to the rescue!

We can define an AppSettings helper like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

public static class AppSettings
{
    public static T Get<T>(string key)
    {
        var appSetting = ConfigurationManager.AppSettings[key];
        if (string.IsNullOrWhiteSpace(appSetting)) throw new AppSettingNotFoundException(key);

        var converter = TypeDescriptor.GetConverter(typeof(T));
        return (T)(converter.ConvertFromInvariantString(appSetting));
    }
}

//usage
var debugSetting = AppSettings.Get<bool>("Debug");

Here we use TypeDescriptor.GetConverter() method to get a TypeConverter for the type T we want to convert to. Then we get the converted value from the converter by calling its ConvertFromInvariantString, the input to which is of course the value from the appSetting.

This helper can now be used for various types which have converters. Consider a sample App.config:

1
2
3
4
5
6
7
8
9
10

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="IsDebug" value="true" />
    <add key="MyEnumValue" value="1" />
    <add key="MyEnumValue2" value="A" />
    <add key="ServiceEndpoint" value="http://www.example.com" />
    <add key="MyFavColor" value="Red" />
  </appSettings>
</configuration>

With that config, you can easily get the values you need, in the type that you will use them in:

1
2
3
4
5

var isDebug = AppSettings.Get<bool>("IsDebug");
var myEnumValue = AppSettings.Get<TestEnum>("MyEnumValue");
var myEnumValue2 = AppSettings.Get<TestEnum>("MyEnumValue2");
var myFavColor = AppSettings.Get<Color>("MyFavColor");
var serviceEndpoint = AppSettings.Get<Uri>("ServiceEndpoint");

The above example works for bool, TestEnum, Color and Uri types. Easy!

Before I start talking about how to go about precompiling razor views in ASP.NET MVC 3, let’s talk about the why.

The foremost reason is performance. Razor views are compiled at runtime. In projects with many views, this can be slow down the app start-up time. For better performance, it is ideal to precompile your razor views.

Even if performance if not a criterion and you don’t have that many views anyway, you also get the advantage of making sure that your razor views do compile at build time rather than at runtime. By default, in MVC 3, you get compile errors from the razor view only when you hit that view from your app. You get no idea, even if you have some silly typo ( though your IDE might be complaining ), till you hit the app. It is nice to have the safety net of compiling your razor views at build time.

Note that precompiling razor views, as will be described here, is NOT the came as compiling them. MVC 3 has the MvcBuildViews feature whereby, by changing the default <MvcBuildViews>false</MvcBuildViews> to true in you csproj file, you can compile your razor views at compile time. But this can only be used for catching compilation errors in your views, and it’s not the same as precompilation. If you don’t want precompilation, you can live with this. It is a simple one line change in your csproj file, and you get the safety net that we talked about. Precompiling your views is slightly more complex. But, MvcBuildViews will slow down your build ( maybe that is the reason why it is not enabled by default!). It is just not as efficient as the precompilation technique that we are going to talk about. In our project, with lots and lots of views, the build time with precompilation was faster than the build time with MvcBuildViews by a whopping 50%. Thats a lot you can shave off your build times.

Another reason to use precompilation is that you don’t need to deploy all those .cshtml files to your web server. This will reduce deployment times, especially, again, for projects with many view files.

Hope these reasons convinced you that precompiled razor views is the best thing since sliced bread. I will talk about the specific approach that I took to achieve this. We will be using the amazing library called RazorGenerator.

Go ahead and install the Nuget packages RazorGenerator.Mvc and RazorGenerator.MsBuild into your MVC project.

The RazorGenerator.Mvc package brings in the PrecompiledMvcEngine and registers it as the view engine in your mvc project. It does so in the RazorGeneratorMvcStart class found under App_Start folder:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

[assembly: WebActivator.PostApplicationStartMethod(typeof(MvcApplication1.App_Start.RazorGeneratorMvcStart), "Start")]

namespace MvcApplication1.App_Start {
    public static class RazorGeneratorMvcStart {
        public static void Start() {
            var engine = new PrecompiledMvcEngine(typeof(RazorGeneratorMvcStart).Assembly) {
                UsePhysicalViewsIfNewer = HttpContext.Current.Request.IsLocal
            };

            ViewEngines.Engines.Insert(0, engine);

            // StartPage lookups are done by WebPages. 
            VirtualPathFactoryManager.RegisterVirtualPathFactory(engine);
        }
    }
}

It uses WebActivator to add the PrecompiledMvcEngine as a view engine in your project. Note also the line UsePhysicalViewsIfNewer = HttpContext.Current.Request.IsLocal. That enables you to develop with precompiled razor views as you would have done before. When you make changes to a razor file, you don’t have to compile your project ( and hence precompiling the razor views ). Just refresh the app and see the change. As is obvious from the line, if the actual .cshtml file are newer, it will use them for local requests, instead of the precompiled version. This means seamless transition to precompile views. Neat!

Now, let’s get to the exciting part - hooking up the actual precompilation of views at build time. This is where the RazorGenerator.MsBuild package comes into the picture. It provides msbuild targets to precompile your razor views that you can hookup into your csproj.

You want to import the RazorGenerator.targets that comes with the package in your csproj:

1
2

<Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" Condition="false" />
<Import Project="$(SolutionDir)\packages\RazorGenerator.MsBuild.1.5.0.0\tools\RazorGenerator.targets"/>

The second import is the one being added, and I add it below the one importing the WebApplication related targets.

Next, you have to add a property that the target imported from the RazorGenerator.targets files uses to check if precompilation is to be done or not:

1
2

<MvcBuildViews>false</MvcBuildViews>
<PrecompileRazorFiles>true</PrecompileRazorFiles>

PrecompileRazorFiles is the property being added and I have added it just below MvcBuildViews in the csproj for obvious reasons.

That’s it! You can now build your application and the views will be precompiled. Try introducing a compile error in one of your razor views and you should get the error when compiling your solution. The precompiled files will be generated under obj\CodeGen.

There is one more thing that you have to do if you don’t want to include the views as part of your deployment package. By default, the .cshtml files will be created as Content type. The web applications target looks for all content types and includes them in the _PublishedWebsites folder. So you have to change the Build Action for your views to None so that they don’t get included. This can be easily automated without changing the view properties. Just add the below to your .csproj

1
2
3
4
5
6
7

<Import Project="$(SolutionDir)\packages\RazorGenerator.MsBuild.1.5.0.0\tools\RazorGenerator.targets"/>
<Target Name="BeforeBuild">
    <ItemGroup>
      <Content Remove="Views\**\*.cshtml" />
      <None Include="Views\**\*.cshtml" />
    </ItemGroup>
</Target>

I am adding the BeforeBuild target right after the previouly added line to import the RazorGenerator.targets file. The BeforeBuild target is a hook provided to do things as a pre-build event. Here we remove all .cshtml from Content itemgroup and add them all to the None group. Now, when the web applications target is executed, the views will not be copied over as they are not Content type anymore.

That’s all there is to it.

Wait, what about using the RazorGenerator Visual Studio extension which generates the backing .generated.cs file when you save the file. I decided not to go with that approach because using the msbuild targets is much cleaner. With the extension approach, not only do you need an extension to visual studio that every team member has to be told to install, all view files, old and new, have to be edited to specify a custom tool. Also, the .generated.cs files pollute you solution while navigating. Too much of an hassle, and I don’t recommend that approach. Using the steps given above, you can achieve precompilation of your razor views without any changes to the way you have been developing so far.

Internet activist, ThoughtWorker and my colleague, Aaron Swartz, committed suicide today. A sad day for us all. I knew that every ThoughtWorker stood behind him and would have done anything to help him. And we will carry forward his flag, and try to make this world a better place to live, much like how he wanted it to be.

I didn’t know him personally. I wish I did. It would have made me a better person. I knew him from his acts and his writings. His writings were profound, and gave much insight into the thought process of a person who contributed to the RSS specification when he was 14 years old. His writings have, and continue to, provide inspiration to young and old alike. I would like to honour him by remembering few of his writings that inspired and moved me and definitely changed my life in more than one way.

How to get a job like mine

In this article, Aaron demystifies and deconstructs his own success, while also talking about the problems faced enroute.

https://aaronsw.jottit.com/howtoget

Raw Nerve

These are a bunch of articles on how to improve life. “Believe you can change”, one of the articles in the series, is a truly thought provoking and inspiring read.

http://www.aaronsw.com/weblog/rawnerve

HOWTO: Be more productive

This article is full of sane advice on how one can be more productive. Simple and clear. It really shows the strengths of his reasonings, and his ability to express them. I have not read a better piece on how to overcome procrastination and bbecome more productive.

http://www.aaronsw.com/weblog/productivity

What Happens in The Dark Knight

Impressive analysis on the Dark Knight story. Even the ardent followers of Nolan and Batman will come out learning something new.

http://www.aaronsw.com/weblog/tdk

His writings will forever continue to inspire and amaze. So will his code and libraries that he wrote like web.py

RIP Aaron Swartz

I love doing git shortlog -sne to get number of commits by each author in a git repo ( don’t aske me why!)

One problem with running it on git repos of long running projects is that authors may change their name or email or both during the course of the project due to many reasons ( most of the times, I have seen people realize few commits into the project that they were using a wrong / personal / professional email id or username. Other times, it is because, being complete newbies to git, they did not configure the username and email and the default ones generated by git were being used.)

When that happens, you can’t get proper counts from the git shortlog output. Here’s a sample output from a dummy repo with two commits. Both are by me, but with different names:

1
2
3

C:\projects\test.git [master]> git shortlog -sne
     1  Manoj <manojlds@gmail.com>
     1  Manojlds <manojlds@gmail.com>

Eventhough I have made 2 commits (yay!), I have 1 commit assigned to each of my names.

How do I fix this and make the git shortlog output more meaningful?

This is where a .mailmap file comes into the picture. It contains the mapping from the “wrong” email ids and / or names to the right ones:

Proper Name <proper@email.xx> Commit Name <commit@email.xx>

The above maps the Commit Name <commit@email.xx> to the Proper Name <proper@email.xx> ( wasn’t that obvious? Yeah, but I had to make it explicit, weren’t I?) There are multiple forms for specifying the correct user name and email in a mailmap. Learn more about git shortlog and .mailmap at the manpage for shortlog

You can just specify the correct name for the email address in the below form:

Proper Name <commit@email.xx>

This is what I need ,as both my commits have the correct email address, but use different user names, and I want to specify the canonical user name. To fix the authors list in my dummy repo and make sure that the 1 commit is also attributed correctly to me, I can write a .mailmap file with the following contents:

Manoj <manojlds@gmail.com>

You can place the file with that name at the root of your repo and the run git shortlog -sne to see that authors list is like we want. You can also set the config mailmap.file with the path to the mailmap file and git will pick up the file from that location if the .mailmap is not found at the root of the repo. You can use this when you don’t want mailmap to be part of your repo.

Our client’s website makes a call to an external service ( internal legacy service, actually ) to get JSON response needed for a feature that is part of a specific page of the website. The feature is based on the country from which the user is accessing the page and displays different information for different countries.

This feature has been in production for a few days now, and all has been good. But we found that, for Norway users, the feature was not being displayed. That happens when there was some problem with accessing or processing the data from the service.

At a cursory glance, the JSON response from the service seemed fine. The data pertinent to the feature were all coming in. The application should be working as expected, like it does for other countries.

Then we noticed that the country part of the JSON data was coming back as “false”. The data that ought to return the country code - like “GB”, “US”, etc. - was returning false.

Some weird bug in the external service it seemed apparent, but it dawned on us what that bug might really be - the country code for Norway is “NO” and it was being serialized as boolean false in the Cold Fusion web service!

Fun little bug there. We do have consumer driven contracts tests against the service, and functional tests, but am not sure how issues like this could have been caught? Yes, the service should have been more thoroughly tested, but what could we, as consumers, have done better? Just smile a bit and file a bug, for now.`

Powershell is amazing. It really lives upto its name. But many a times, I have found new comers tripping on a few Powershell oddities. Here I higlight my favorite three.

First and foremost, something that confuses most anyone who starts using Powershell is what version of Powershell that they are using. Micorosft, in its usual ways, install Powershell, upto the latest 3.0 version, at the C:\Windows\System32\WindowsPowerShell\v1.0 ( you can get it by doing $pshome from a Powershell console.) Notice the v1.0 at the end. Seeing that, it obviously raises confusion as most resource on Powershell is for v2.0 ( at the time of this writing, eventhough v3.0 is out.) ad people want to be on v2.0 and not v1.0. The easiest way to identify the Powershell version is to just run get-host in Powershell. Or to be more specific (get-host).Version.

Another thing that causes even more confusion is the way to pass arguments to functions.

Say there is a function as defined below:

1
2
3
4

function Name($firstName, $lastName){
    write-host "First name: $firstName"
    write-host "Last name: $lastName"
}

If we call the function Name John, Smith, the expected output would be something like:

1
2

First name: John
Last name: Smith

But what would be seen is:

1
2

First name: John Smith
Last name:

I have seen many do this mistake. Arguments in Powershell, to Powershell functions are separated by space. When you comma delimit them, you are actually passing an array of those values. In this case, the array with two values ("John", "Smith"), was given to the parameter$firstName` and hence the output you see.

I mention Powershell functions above because method calls on .NET objects still use comma to delimit arguments:

1

[string]::format("{0},{1}", "comma", "delimited")

Lastly, I see newcomers get trumped by Select-Object. As known, Select-Object selects particular properties from objects and sends them to the pipeline.

For example,

1

$names = get-childitem . | select-object Name

would put the names of directories and files in the current directory in $names

It is logical to thing that $names is a collection of strings. But it is not! It is a collection of objects with a single property (in this case) Name. To access the actual name, you do $names[0].Name.

If you wanted only the collection of string in the first place, you use the -ExpandProperty parameter (-expand in short) of Select-Object:

1

$names = get-childitem . | select-object -expand Name

Now, $names is just a collection of vanilla strings.