Archive

Archive for the ‘Source Control’ Category

Mercurial Server using hgweb.cgi on Ubuntu

June 30, 2011 Leave a comment

In a previous post, we set up a virtual machine template for an ubuntu server. Now that we have set up a clone of this machine, it is time to set up a Mercurial repository server

Abstract

Mercurial provides an easy to use repository server via a python cgi script. Mercurials protocol facilitates fast transfers over http, making it superior over an ssh solution based (such as git) when considering minimal protocol overhead vs. ease of use. As my webserver of choice, we will use lighttpd. This guide will follow the instructions published here http://mercurial.selenic.com/wiki/PublishingRepositories#multiple

Installing Python

This one is simple:

ubuntu@localhost: sudo apt-get install python

Installing Lighttpd

To install lighttpd, run:

ubuntu@localhost: sudo apt-get install lighttpd

Next, we need to create a specific configuration for our mercurial cgi script. We need to redirect all incoming requests to the cgi script, and then we apply some url rewrite magic to remove the ugly hgweb.cgi from our URLs. The hgweb.cgi script will be served from /var/www/hgweb.cgi. If you use a different location, make sure to chown it to www-data and chmod+x it (all described in the mercurial wiki). I created my config like this:

ubuntu@localhost:~$ sudo vi /etc/lighttpd/hg.conf
ubuntu@localhost:~$ cat /etc/lighttpd/hg.conf 
url.rewrite-once = (
  "^([/?].*)?$" => "/hgweb.cgi$1",
   "^/([/?].*)?$" => "/hgweb.cgi$1"
)

$HTTP["url"] =~ "^/hgweb.cgi([/?].*)?$" {
             server.document-root = "/var/www/"
             cgi.assign = ( ".cgi" => "/usr/bin/python" )
}

Next is the lighttpd config, that will need to reference our hg.conf and enable mod_cgi:

ubuntu@localhost:~$ cat /etc/lighttpd/lighttpd.conf 
include "hg.conf"

server.modules = (
	"mod_access",
	"mod_alias",
	"mod_compress",
 	"mod_redirect",
        "mod_rewrite",
        "mod_cgi"
)

Further configuration Tricks

You should force hgweb.cgi to serve UTF-8 content. Fortunately enough, this is as simple as adding (or uncommenting) the following lines to hgweb.cgi:

import os
os.environ["HGENCODING"] = "UTF-8"

You will also need a hgweb.conf right next to hgweb.cgi and reference it from there (again, described in the mercurial wiki). For reference, my configuration includes all repos sourced under /var/hg/repos (and subdirectories) and allows anonymous push (I’m authenticated via VPN policy):

ubuntu@localhost:~$ cat /var/www/hgweb.config
[paths]
/ = /var/hg/repos/**

[web]
baseurl = /
allow_push = *
push_ssl = false

Final Words

That’s all there is to it. To make the server available via DNS, you need to make sure the servers hostname is registered with your local DNS. In my case, I simply added a static record to it.

Version Control Tooling List

April 20, 2010 Leave a comment

A short list of the VCS tools I’m currently using. Maybe it’s useful for anybode else than me searching for a hint.

Mercurial:

  • TortoiseHg (Windows), Murky (Mac OS),  Command Line only on Linux. For more complex operations (e.g. using Mercurial Queues) I do use the command line.
  • Hosting: GoogleCode, bitbucket. Bitbucket looks nicer and has a great community, google code is more reliable.

Git:

  • CommandLine only in most cases, gitk for repository browsing. GitX is great for Mac OS. Will check QGit soon.
  • Hosting: Github. (There’s no more to say).

Subversion:

  • SmartSVN, IDE Integration (XCode, VisualStudio(AnkhSvn), Eclipse, MonoDevelop).
  • Hosting: Supported on almost every hosting platform, Google Code wins for me with its clear UI and reliable service.

For Windows, the Tortoise(X) explorer shell integrations are a good choice, however right now I’m only using TortoiseHg as there may be some clashes when I’m using e.g. hg (for private commits) and svn on one project. As a diff/merge tool I’m using the excellent DiffMerge from SourceGear.

Categories: Source Control, Tools

Subversion Client Evaluation

April 19, 2010 2 comments

Even though I am a huge believer in DVCS, for some projects I am still bound to use subversion.  I have development environments set up under Windows (primarily .NET), Mac OS (Mono and iPhone) and Linux (Mono, Haskell), so I’d like my tools to be nearly identical on all three platforms to reduce any friction not directly related to writing code.

I use version control on any project I do, some of them being cross-platform projects (e.g. mono ports), so having frictionless access to my VCS on any platform is of primary concern to me. All version control systems I happen to use at the moment run on all platforms and have good (svn, git) or excellent (hg) command line utilities, but I like the comfort of a GUI. Especially for browsing history and diffing or merging, graphical tools are irreplaceable.

Finding a good multi-platform VCS Client GUIs for subversion is not very easy, I have evaluated quite a few of them but haven’t reached any final conlusion. Here are my notes on each:

eSVN:
Qt based, only very basic repository operations, no stable version available, development has stalled in 2007. No recent releases (SVN 1.6.x) available.

QSvn:
Qt based, lightweight and solid. Actively developed, recent releases available. Easy project set-up and configuration. The GUI is very streamlined and sufficient for day to day use. Very intuitive. No merge support.

RapidSVN:
WxWidgets based, development has stalled in late 2009. There might be future maintenance but so far it doesn’t look very promising. Ugly GUI. Haven’t tried any further.

Subcommander:
Qt based, rich feature set and actively developed. The user interface is not very intuitive but you’ll get used to it. I found refreshing the repository status to be very slow. So far the most complete OSS choice (including merge support). I’d prefer QSvn for the standard tasks.

SmartSVN:
Java based, sadly enough it’s commercial. You’ll get a 30-days trial for the pro version, after that only features of the “Foundation” version are enabled. I have found it to be very intuitive, stable and fast. SmartSVN has some features that will make life with SVN easier such as prepared commits (think of a private patch queue, unfortunately not in the free edition) and a graphical revision graph. Very good merge support.

In principle I am a fan of stand-alone utilities as I think of coding and version control as sequential tasks. Having to collect my changes manually forces me to review them once more before finally comitting, which is a good thing. Subversion is a very mature VCS and has been around for very long, so I expected to find some decent OSS Clients for it. To be honest, the existing projects seem to be far behind here. The commercial SmartSVN is the best stand-alone client I have seen.

Given this situation, I am heavily leaning towards using IDE integrated tools for subversion. In contrast to a DVCS, using a centralized system takes away more than enough freedom anyway, so using IDE integrated tools won’t hurt any further. Most of the IDEs I use provide native Subversion support (XCode, MonoDevelop, Eclipse) or via a plugin (AnkhSVN for Visual Studio). The supported features are equivalent to what a simple tool like QSvn has to offer, for everything else I’ll probably use SmartSVN.

Setting up a Mercurial Repository Server using hgwebdir.cgi on IIS6

March 28, 2010 1 comment

My exams are over now and I will hopefully have enough time to cross some points off my personal to-do list. One thing I have always wanted to do is setting my home server up to let it serve my Mercurial Repositories via http. Given the distributed nature of Mercurial I don’t find myself in the situation to access my repos remotely but it can be quite handy from times to times. In the past I have accessed them via VPN (using OpenVPN), having them sit on a Windows Network Share. Needless to say performance was suboptimal but actually quite acceptable if my internet connection was fast and stable. However, accessing it via 3G is a completely different story.

When Google Code evaluated their plans to support at least one major DVCS, they did a detailed analysis with a special focus on supported protocols and performance. They finally decided to support Mercurial because it has a very efficient HTTP protocol implementation that was a good fit with Google’s Infrastructure. Besides supporting file shares and HTTP, Mercurial does also support SSH, which is not an option for me since my server runs a Microsoft Stack. There’s a good list of supported protocols and sharing methods on the Mercurial Wiki. Since I want to share multiple repositories, I decided to go with the hgwebdir solution. Basically it is just a smallish Python CGI application that can also serve a nice repository browser etc.

Set up is fairly easy, I found Jeremy Skinners tutorial very helpful, even though it’s using IIS 7 (I’m running IIS 6).  What caused me some headaches, was implementing a nice URL rewriting. Because all access is done via hgwebdir.cgi, it’s an ugly part of the URL like so:

   http://myserver/hg/hgwebdir.cgi/LDT-trunk/

The Idea with using a URL rewriting is to let the user access repositories via a clean URL like http://myserver/hg/LDT-trunk/,which is then rewritten to the former. Hgwebdir.cgi supports this method because one can change the way it generates links on the web interface by providing a new base-Url  (/hg instead of /hg/hgwebdir.cgi). This and URL Rewriting for Apache using a .htaccess file is described on the Mercurial Wiki.

To be honest, it’s a fairly optional step but being the perfectionist I am I couldn’t resist to try it with IIS. Jeremy Skinner proposes Microsoft’s official URL Rewrite, but that’s only compatible with IIS 7.  The best tool I could find for IIS 6 is the excellently documented Ionic’s ISAPI Rewrite Filter, which features a very similar and powerful regular expression based configuration as Apache .htaccess. Set up was very fast but getting the configuration right was not very easy and took a little debugging. The IIRF tooling (TestDriver.exe) was really helpful here. Here’s my IIRF.ini for your reference:

# Send requests for files that exist to those files.
RewriteCond %{REQUEST_FILENAME} !-f
# Send requests for directories that exist to those directories.
RewriteCond %{REQUEST_FILENAME} !-d

# prevents processing of already correct urls
RewriteRule ^/hg/hgwebdir.cgi/(.*)$ - [L,I]
# redirect every request to the hgwebdir.cgi script
RewriteRule ^/hg/(.*)$ /hg/hgwebdir.cgi/$1
# redirect root directory access to the website root
RewriteRule ^/hg /hg/hgwebdir.cgi/

Everything was working fine, except the URL rewriting didn’t work out for URL’s that contained encoded Umlauts (like Ü becomes %DC). On top of that, file contents weren’t displayed using the correct encoding (which is UTF-8). The latter was easy to fix by simply uncommenting the first two lines in hgwebdir.cgi that will set the HGENCODING environment variable, which tells Mercurial what encoding to use, to UTF-8:

import os
os.environ["HGENCODING"] = "UTF-8"

Mercurial internally makes no assumptions about the file contents or file types it tracks, so in this case encoding is simply a matter of presentation. All other internal data (changelogs, tags etc.) are stored as UTF-8 according to the wiki. There are still some minor anomalies in the web interface, but I will investigate them later.

The issue with incorrect URL rewritten URL’s was much harder to track down. I used Fiddler to analyse the GET/RESPONSE stream but couldn’t find anything that was obviously wrong. The IIRF log contained the following lines:

Sun Mar 28 16:31:57 -  3260 - DoRewrites: Url (decoded): '/hg/LDT-trunk/file/c2e75c0ef33c/src/Übc.xx'
Sun Mar 28 16:31:57 -  3260 - DoRewrites: Rewrite Url to: '/hg/hgwebdir.cgi/LDT-trunk/file/c2e75c0ef33c/src/Übc.xx'

That looked somewhat unsuspicious to me, until I remembered that special characters in URL’s need to be specially encoded. By default IIRF attempts to decode the URLs when a request is received and does only process the decoded URLs (hence the small hint in the log file). This behavior can easily be configured via the IIRF configuration using UrlDecoding Off.

Categories: Source Control, Tools

Configuring Git and Mercurial to use DiffMerge

January 28, 2010 1 comment

My preferred merge/diff tool at the moment is DiffMerge by SourceGear. Besides normal diff and also folder diffs, it supports three-way merging and has a pretty sweet, intuitive UI with a-well chosen colour scheme and excellent shortcuts. The reason it is my tool du jour is its perfect cross-platform implementation supporting Windows, MacOS X and Unix. The feature set (and the UI!) is same on all platforms. Remembering only the workings of one diff tool significantly reduces friction for me.

Integrating DiffMerge with the various VCS I use (mercurial, git and svn) is quite cumbersome, but is fortunately very well possible. I’ll post my sample configurations here. The configuration files are alway located in the user’s home directory. Configuration is the same for Windows/Mac OS X and Unix, however you’ll have to adapt the path to the DiffMerge executable accordingly.

Mercurial (~/Mercurial.ini):

[diff]
git = True

[extensions]
hgext.extdiff =

[ui]
merge = diffmerge

[extdiff]
cmd.diffmerge = c:\Programme\SourceGear\DiffMerge\DiffMerge.exe

[tortoisehg]
vdiff = diffmerge

[merge-tools]
diffmerge.executable = c:\Programme\SourceGear\DiffMerge\DiffMerge.exe
diffmerge.args = --result=$output -t1="Local Version" -t2=$output -t3="Other Version" --caption=$output  $local $base $other
diffmerge.binary = False
diffmerge.symlinks = False
diffmerge.gui = True
diffmerge.premerge = True

Mercurial will fire up DiffMerge during merges automatically from now on. Since the built-in diff command will always print directly to console (you can pipe this into a patch file too, very useful!) you need the extdiff extension to enable diffing with DiffMerge. That’s why TortoiseHg also needs to be configured explicitly to use DiffMerge as visual diff tool.

Git (~/.git):

[mergetool "diffmerge"]
        cmd = diffmerge --merge --result=$MERGED $LOCAL $BASE $REMOTE
        trustExitCode = true
[diff]
        tool = diffmerge
[difftool "diffmerge"]
        cmd = diffmerge \"$LOCAL\" \"$REMOTE\"

Git does not merge automatically but will rather leave merge markes in the file. You have to use

git mergetool -t diffmerge

to bring up DiffMerge. On the web one can find recommendations to set

trustExitCode = false

when using DiffMerge. Since v 3.1. this is no longer needed because DiffMerge now returns correct status codes indicating whether a merge was successful or failed.

Categories: Source Control, Tools
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: