git worktree - making TYPO3 Core development more efficient

Let me start with a few words on how I set up my environment to develop for TYPO3 CMS Core. I use Windows as my main operating system and as my development platform, yet I dislike running the webstack natively in Windows. Therefore I use a virtual machine in VMware Player hosting my MySQL and Apache HTTP server. The software sources, which I work on - like TYPO3 Core - are stored in Windows and are mounted into the VM using HGFS for shared usage and easy debugging.

Since I commonly need to test patches on multiple versions of TYPO3, I used to have a dedicated clone for each version (each version is a branch). That means I had to have 4 copies of the whole GIT repository of the TYPO3 Core on my harddisk. That is roughly 50K files making up more than a gig. So far so good, a gig shouldn't really matter. But things got really annoying when it comes to pulling in the latest changes (you have to download the new changes 4 times) or even worse, when backporting patches.

Back then I had to do many backports of patches (i.e. adjusting a patch suitable for the latest version for an older version and applying it there) and that got cumbersome the more the newer version (CMS 7) diverged from the former versions.
Backporting a patch manually meant to execute one of two procedures:

1. Pull in the change on current master, checkout the other branch, cherry-pick the patch from master.
   The advantage is that the cherry-pick is rather easy, but the disadvantages prevail. Switching the branch causes a few hundred files to be changed and PHPStorm has to update their index each time. Additionally my TYPO3 test-instance connected to that repository is actually the one for master, hence I couldn't tests the backported patch this way.
2. Cherry-Pick the patch from Gerrit (the review system) directly onto my checkout for the backport target version.
   The benefit compared to the first procedure is that I don't need to switch branches and my test-instance is the right one, but the problem here is that cherry-picking actually downloads the affected history between the patch on master and the current state of the backport branch (which is possibly a few hundred commits behind), which is needed to calculate the correct way to apply the patch. You don't want to hang on a slow mobile connection when doing such things.

As one can see, neither procedure is really satisfying and therefore a better solution was really welcome. Luckily GIT 2.5 ships this awesome feature called git worktree.

git worktree

The git worktree command allows to run multiple checkouts of different branches from a single repository at the same time. One can have a folder for each branch sitting right next to each other in the same parent directory.

The history is kept only in the original clone directory. All other worktrees only store a reference to the "mother" and that's it.

One can use the usual git commands in each worktree to commit, merge, rebase, etc and even better, one can cherry-pick patches from different branches of course.

Using it with TYPO3

This is exactly what I needed to have my independent TYPO3 version source folders and still have the full power of working only on a single repository - sharing commit hooks, configs, etc. Each TYPO3 testing instance can point to its source folder, each source folder is its own PHPStorm project. (PHPStorm support git worktrees since version 2016.1)

My folder (repository) setup now looks like this:

TYPO3 (main repository)
TYPO3_4-5 (worktree)
TYPO3_6-2 (worktree)
TYPO3_7-6 (worktree)

The commands to achieve this are pretty straight forward:

# clone the repository into the TYPO3 folder as described in the contribution walk-through tutorial
....
git clone .... TYPO3
# change into the repository
cd TYPO3
# add a new worktree, syntax is: git worktree add <path> <branch>
git worktree add ../TYPO3_7-6 TYPO3_7-6
git worktree add ../TYPO3_6-2 TYPO3_6-2
....

Conclusion

Working with the worktrees is really easy and painless. Cherry-picking patches from one to the other branch is as fast as it can be for a local repository. An extra benefit is that you only need to pull/fetch from origin once for all worktrees as updating the worktrees is a matter of running a git merge or git reset command.

There is a tiny pitfall though, which has to be remembered. Do not git pull in multiple worktrees at the same time, it will not work. The first process will lock the repository. Pull only in one worktree (it does not matter which one) and only git merge in the other ones.