How to check the status of all git repositories at once? - windows

Introduction
In order to check the status of git repositores, git status could be issued from the root of a repository.
C:\path\to\git_repositories\git_repo_1>git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
If a directory consists of multiple, e.g. 50 git repositories
C:\path\to\git_repositories>dir
Directory of C:\path\to\git_repositories
.ssh
git_repo_1
...
git_repo_50
0 File(s)
51 Dir(s)
Nor
C:\path\to\git_repositories>git status .
fatal: Not a git repository (or any of the parent directories): .git
neither
C:\path\to\git_repositories>git status ./.
fatal: Not a git repository (or any of the parent directories): .git
is able to check the status of all repositories
Question
How to check the status of all git repositories at once?

You could use a for loop that changes into each directory, does git status and then changes back up:
for /f "tokens=*" %a in ('dir /ad /b') do cd %a & git status & cd ..
You need to double the percentages if you use this in a batch file:
for /f "tokens=*" %%a in ('dir /ad /b') do cd %%a & git status & cd ..
Edit:
As suggested by Cupcake, you could do this instead:
for /f "tokens=*" %a in ('dir /ad /b') do git --git-dir=%a/.git --work-tree=%a status
This feels like a more robust and flexible solution (e.g. you could adapt it more easily to work with a list of paths stored in a text file).

I went with:
find . -name .git -execdir bash -c 'echo -en "\033[1;31m"repo: "\033[1;34m"; basename "`git rev-parse --show-toplevel`"; git status -s' \;
I think it's nicer because treats directories recursively.
Edited to show the name of the repo in a cleaner way.

If you are a kind of a tool-guy, you could use a little helper called RepoZ I wrote recently (and still write).
I answered a quite similar question in more detail here.
Here's a screenshot from the current development stage so hopefully you can see whether it seems helpful to you or not:
If you are wondering what that strings like +45 ~403 -88 mean - they are condensed status strings telling you whether there are commits to fetch or push and whether there are added/modified/deleted files locally. More detail on the project site on GitHub

As of 2.34 the for-each-repo command allows running git commands across a list of repos defined in a multi-valued config.
Example config:
[myRepos]
repo = /full/path/without/expansion/repo1
repo = ../can/be/relative/repo2
[alias]
all = for-each-repo --config=myRepos.repo
Then you can run git all status.

Borrowed from coderwall - git status on all repos in folder:
find . -maxdepth 1 -mindepth 1 -type d -exec sh -c '(echo {} && cd {} && git status -s && echo)' \;
find . : to find everything in the current folder
-maxdepth 1 : so that it doesn't recurse into subdirs of the repos
-mindepth 1 : so that it skips the current directory (of depth 0)
-type d : only find directories
-exec sh -c : spawn a shell and give it a command
'(echo {} && cd {} && git status && echo)' : the command given to the shell
echo {} : echo the directory found by find
cd {} : cd into the directory found by find
git status -s : run the actual git status, with the -s (short) option
echo : echo an empty line, for readability
\; : semicolon to run shell for each of the found directories instead of passing them all to one shell as arguments
Also, you could make an alias in your bash_profile use the following with escaping to make it work:
alias gs_recursive='find . -maxdepth 1 -mindepth 1 -type d -exec sh -c "echo {}; cd {}; git status -s; echo" \;'

If you have a system that is made up of numerous Git repositories, that is possibly a situation for which the Google repo tool was created: https://code.google.com/archive/p/git-repo/
If you find yourself writing scripts to distribute Git commands over these repositories, you should probably switch to repo.
repo works with an (unfortunately) XML file called the "manifest.xml" which specifies the Git repositories that are in the workspace. For each repo specifies which branch and commit should appear; you can use repo to "pin" the Git repositories to particular versions, or to track heads.
If you have a repo workspace then
$ repo status
then repo will do pretty much what you're asking for: iterate over the Git repos and give you a status.

Oof... git's design failed.
When I looked up this question, I was reminded of all the examples out there that use cd to jump into repos and then run git commands in there.
You need to use the options --git-dir= and --work-tree=
(Well... that's what I want to say anyway.)
When we use those options --git-dir is the .git directory in the repo and the --work-tree= value is the repo's root directory.
For example
Let's say we are in C:\, the repo we're interested in is at C:\workspace\foobar
To get it's status without doing barbaric cd manuevers:
git --git-dir=c:\workspace\foobar\.git --work-tree=c:\workspace\foobar\ status
or for the nice short status we all like.
git --git-dir=c:\workspace\foobar\.git --work-tree=c:\workspace\foobar\ status -s
So that's only a little more than double the typing, and we get to use some barely coherent git options.
Now that we know the civilised way...
What's more we haven't lowered ourselves to virtually banging rocks together and typing...
cd c:\workspace\foobar; git status -s
Why would we ever do that.
(seriously though, do the cd. It's very rare to even find a script doing the --git-dir/--work-tree nonsense. It's extremely poor user service and has all the hallmarks of 0% design thought.)
Now let's do reflog... (ok let's not.)

Related

How to zip and ignore all files specified by .gitignore

I have a few dozen git repositories under one directory, all of them contain not only source code but also build artifact, developer artifact, blobs and other data normally ignored by .gitignore
I would like to quickly transfer these source code to another computer. Is there an easy way to package everything that is not ignored by git?
To add more context:
I have 50-ish git folders or so, each on different branches with some current changes
I would like to copy all files that are tracked by git, at the current state ( current branch plus current local change )
If by cloning them all, it means to go to each folder, commit the current changes locally, remember the branch name, git clone local branch to another folder, do that 50 times then I am strongly agaist it
One-liner:
$ alias gitzip="git archive HEAD -o ${PWD##*/}.zip"
$ cd mydir
$ gitzip
If you want to include history, use git bundle (or xeyownt/git-subundle if your repositories have submodules)
See this script as an example:
for repository in ${repositories[#]}
do
echo "$repository..."
declare file_name="../$repository-$(git rev-parse --short HEAD).bundle"
cd $repository
if [ -f $file_name ]; then
echo 'Found bundle is current'
else
git bundle create $file_name --all
fi
cd -
done
Each repository gives one file (without anyprivate or ignored files), that you can easily copy and, once copied, cloned from.
You can try this :
for dir in repo1 repo2 repo3; do
git -C "$dir" ls-files | perl -pe "s[^][$dir/]"
done | zip /tmp/test.zip -#

How to make a copy of all modified files in Mercurial working copy

How can I make a copy of all uncommited modified or added files in a Mercurial working copy?
Obviously I can go to each directory that has a changed file and copy it. I am looking for a process I can do from the command line (Windows or Powershell) or using TortoiseHg.
Thanks
Matthew
You can use purge alias from hgrc man page as good starting point and
Change status command accordingly to your needs: hg status --no-status -m
Pipe output to xargs's cmd.exe oneliner equivalent hg status ... > filelist.txt && FOR /F %k in (filelist.txt) DO copy %k \BACKUPPATH\%k (status without -0 parameter) or install any xargs port (as part of GOW, for example) and use original form of alias status -0 ...|xargs -0 ...

Git symbolic links in Windows

Our developers use a mix of Windows and Unix-based OSes. Therefore, symbolic links created on Unix machines become a problem for Windows developers. In Windows (MSysGit), the symbolic link is converted to a text file with a path to the file it points to. Instead, I'd like to convert the symbolic link into an actual Windows symbolic link.
The (updated) solution I have to this is:
Write a post-checkout script that will recursively look for "symbolic link" text files.
Replace them with a Windows symbolic link (using mklink) with the same name and extension as dummy "symbolic link"
Ignore these Windows symbolic links by adding an entry into file .git/info/exclude
I have not implemented this, but I believe this is a solid approach to this problem.
What, if any, downsides do you see to this approach?
Is this post-checkout script even implementable? I.e., can I recursively find out the dummy "symlink" files Git creates?
Update note
For most Windows developers struggling with symlinks and git on Windows and the issues of sharing a repo with *nix systems, this topic is a solved problem -- once you update your Windows understanding of mklink a bit and turn on Developer Mode.
See this more modern answer before digging into the following deep git hacks discussion.
Older systems:
I was asking this exact same question a while back (not here, just in general), and ended up coming up with a very similar solution to OP's proposition.
I'll post the solution I ended up using.
But first I'll provide direct answers to OP's 3 questions:
Q: "What, if any, downsides do you see to this approach?"
A: There are indeed a few downsides to the proposed solution, mainly regarding an increased potential for repository pollution, or accidentally adding duplicate files while they're in their "Windows symlink" states. (More on this under "limitations" below.)
Q: "Is this post-checkout script even implementable? i.e. can I recursively find out the dummy "symlink" files git creates?"
A: Yes, a post-checkout script is implementable! Maybe not as a literal post-git checkout step, but the solution below has met my needs well enough that a literal post-checkout script wasn't necessary.
Q: "Has anybody already worked on such a script?"
A: Yes!
The Solution:
Our developers are in much the same situation as OP's: a mixture of Windows and Unix-like hosts, repositories and submodules with many git symlinks, and no native support (yet) in the release version of MsysGit for intelligently handling these symlinks on Windows hosts.
Thanks to Josh Lee for pointing out the fact that git commits symlinks with special filemode 120000. With this information it's possible to add a few git aliases that allow for the creation and manipulation of git symlinks on Windows hosts.
Creating git symlinks on Windows
git config --global alias.add-symlink '!'"$(cat <<'ETX'
__git_add_symlink() {
if [ $# -ne 2 ] || [ "$1" = "-h" ]; then
printf '%b\n' \
'usage: git add-symlink <source_file_or_dir> <target_symlink>\n' \
'Create a symlink in a git repository on a Windows host.\n' \
'Note: source MUST be a path relative to the location of target'
[ "$1" = "-h" ] && return 0 || return 2
fi
source_file_or_dir=${1#./}
source_file_or_dir=${source_file_or_dir%/}
target_symlink=${2#./}
target_symlink=${target_symlink%/}
target_symlink="${GIT_PREFIX}${target_symlink}"
target_symlink=${target_symlink%/.}
: "${target_symlink:=.}"
if [ -d "$target_symlink" ]; then
target_symlink="${target_symlink%/}/${source_file_or_dir##*/}"
fi
case "$target_symlink" in
(*/*) target_dir=${target_symlink%/*} ;;
(*) target_dir=$GIT_PREFIX ;;
esac
target_dir=$(cd "$target_dir" && pwd)
if [ ! -e "${target_dir}/${source_file_or_dir}" ]; then
printf 'error: git-add-symlink: %s: No such file or directory\n' \
"${target_dir}/${source_file_or_dir}" >&2
printf '(Source MUST be a path relative to the location of target!)\n' >&2
return 2
fi
git update-index --add --cacheinfo 120000 \
"$(printf '%s' "$source_file_or_dir" | git hash-object -w --stdin)" \
"${target_symlink}" \
&& git checkout -- "$target_symlink" \
&& printf '%s -> %s\n' "${target_symlink#$GIT_PREFIX}" "$source_file_or_dir" \
|| return $?
}
__git_add_symlink
ETX
)"
Usage: git add-symlink <source_file_or_dir> <target_symlink>, where the argument corresponding to the source file or directory must take the form of a path relative to the target symlink. You can use this alias the same way you would normally use ln.
E.g., the repository tree:
dir/
dir/foo/
dir/foo/bar/
dir/foo/bar/baz (file containing "I am baz")
dir/foo/bar/lnk_file (symlink to ../../../file)
file (file containing "I am file")
lnk_bar (symlink to dir/foo/bar/)
Can be created on Windows as follows:
git init
mkdir -p dir/foo/bar/
echo "I am baz" > dir/foo/bar/baz
echo "I am file" > file
git add -A
git commit -m "Add files"
git add-symlink ../../../file dir/foo/bar/lnk_file
git add-symlink dir/foo/bar/ lnk_bar
git commit -m "Add symlinks"
Replacing git symlinks with NTFS hardlinks+junctions
git config --global alias.rm-symlinks '!'"$(cat <<'ETX'
__git_rm_symlinks() {
case "$1" in (-h)
printf 'usage: git rm-symlinks [symlink] [symlink] [...]\n'
return 0
esac
ppid=$$
case $# in
(0) git ls-files -s | grep -E '^120000' | cut -f2 ;;
(*) printf '%s\n' "$#" ;;
esac | while IFS= read -r symlink; do
case "$symlink" in
(*/*) symdir=${symlink%/*} ;;
(*) symdir=. ;;
esac
git checkout -- "$symlink"
src="${symdir}/$(cat "$symlink")"
posix_to_dos_sed='s_^/\([A-Za-z]\)_\1:_;s_/_\\\\_g'
doslnk=$(printf '%s\n' "$symlink" | sed "$posix_to_dos_sed")
dossrc=$(printf '%s\n' "$src" | sed "$posix_to_dos_sed")
if [ -f "$src" ]; then
rm -f "$symlink"
cmd //C mklink //H "$doslnk" "$dossrc"
elif [ -d "$src" ]; then
rm -f "$symlink"
cmd //C mklink //J "$doslnk" "$dossrc"
else
printf 'error: git-rm-symlink: Not a valid source\n' >&2
printf '%s =/=> %s (%s =/=> %s)...\n' \
"$symlink" "$src" "$doslnk" "$dossrc" >&2
false
fi || printf 'ESC[%d]: %d\n' "$ppid" "$?"
git update-index --assume-unchanged "$symlink"
done | awk '
BEGIN { status_code = 0 }
/^ESC\['"$ppid"'\]: / { status_code = $2 ; next }
{ print }
END { exit status_code }
'
}
__git_rm_symlinks
ETX
)"
git config --global alias.rm-symlink '!git rm-symlinks' # for back-compat.
Usage:
git rm-symlinks [symlink] [symlink] [...]
This alias can remove git symlinks one-by-one or all-at-once in one fell swoop. Symlinks will be replaced with NTFS hardlinks (in the case of files) or NTFS junctions (in the case of directories). The benefit of using hardlinks+junctions over "true" NTFS symlinks is that elevated UAC permissions are not required in order for them to be created.
To remove symlinks from submodules, just use git's built-in support for iterating over them:
git submodule foreach --recursive git rm-symlinks
But, for every drastic action like this, a reversal is nice to have...
Restoring git symlinks on Windows
git config --global alias.checkout-symlinks '!'"$(cat <<'ETX'
__git_checkout_symlinks() {
case "$1" in (-h)
printf 'usage: git checkout-symlinks [symlink] [symlink] [...]\n'
return 0
esac
case $# in
(0) git ls-files -s | grep -E '^120000' | cut -f2 ;;
(*) printf '%s\n' "$#" ;;
esac | while IFS= read -r symlink; do
git update-index --no-assume-unchanged "$symlink"
rmdir "$symlink" >/dev/null 2>&1
git checkout -- "$symlink"
printf 'Restored git symlink: %s -> %s\n' "$symlink" "$(cat "$symlink")"
done
}
__git_checkout_symlinks
ETX
)"
git config --global alias.co-symlinks '!git checkout-symlinks'
Usage: git checkout-symlinks [symlink] [symlink] [...], which undoes git rm-symlinks, effectively restoring the repository to its natural state (except for your changes, which should stay intact).
And for submodules:
git submodule foreach --recursive git checkout-symlinks
Limitations:
Directories/files/symlinks with spaces in their paths should work. But tabs or newlines? YMMV… (By this I mean: don’t do that, because it will not work.)
If yourself or others forget to git checkout-symlinks before doing something with potentially wide-sweeping consequences like git add -A, the local repository could end up in a polluted state.
Using our "example repo" from before:
echo "I am nuthafile" > dir/foo/bar/nuthafile
echo "Updating file" >> file
git add -A
git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# new file: dir/foo/bar/nuthafile
# modified: file
# deleted: lnk_bar # POLLUTION
# new file: lnk_bar/baz # POLLUTION
# new file: lnk_bar/lnk_file # POLLUTION
# new file: lnk_bar/nuthafile # POLLUTION
#
Whoops...
For this reason, it's nice to include these aliases as steps to perform for Windows users before-and-after building a project, rather than after checkout or before pushing. But each situation is different. These aliases have been useful enough for me that a true post-checkout solution hasn't been necessary.
References:
http://git-scm.com/book/en/Git-Internals-Git-Objects
http://technet.microsoft.com/en-us/library/cc753194
Last Update: 2019-03-13
POSIX compliance (well, except for those mklink calls, of course) — no more Bashisms!
Directories and files with spaces in them are supported.
Zero and non-zero exit status codes (for communicating success/failure of the requested command, respectively) are now properly preserved/returned.
The add-symlink alias now works more like ln(1) and can be used from any directory in the repository, not just the repository’s root directory.
The rm-symlink alias (singular) has been superseded by the rm-symlinks alias (plural), which now accepts multiple arguments (or no arguments at all, which finds all of the symlinks throughout the repository, as before) for selectively transforming git symlinks into NTFS hardlinks+junctions.
The checkout-symlinks alias has also been updated to accept multiple arguments (or none at all, == everything) for selective reversal of the aforementioned transformations.
Final Note: While I did test loading and running these aliases using Bash 3.2 (and even 3.1) for those who may still be stuck on such ancient versions for any number of reasons, be aware that versions as old as these are notorious for their parser bugs. If you experience issues while trying to install any of these aliases, the first thing you should look into is upgrading your shell (for Bash, check the version with CTRL+X, CTRL+V). Alternatively, if you’re trying to install them by pasting them into your terminal emulator, you may have more luck pasting them into a file and sourcing it instead, e.g. as
. ./git-win-symlinks.sh
You can find the symlinks by looking for files that have a mode of 120000, possibly with this command:
git ls-files -s | awk '/120000/{print $4}'
Once you replace the links, I would recommend marking them as unchanged with git update-index --assume-unchanged, rather than listing them in .git/info/exclude.
2020+ TL;DR Answer
Enable "Developer Mode" in Windows 10/11 -- gives mklink permissions
Ensure symlinks are enabled in git with (at least) one of
System setting: check the checkbox when installing msysgit
Global setting: git config --global core.symlinks true
Local setting: git config core.symlinks true
Be careful, support for symlinks in git on Windows is relatively new.
There are some bugs that still affect some git clients.
Notably, symlinks with relative (..) paths are mangled in some programs because of a (fixed) regression in libgit2.
For instance, GitKraken is affected by this because they are waiting on nodegit to update libgit2 from v0.x (regression) to v1.x (fixed).
Recreate missing/broken symlinks
Various levels of success have been reported across multiple git clients with one of these (increasingly forceful and "dangerous") options
Checkout: git checkout -- path/to/symlink
Restore (since git v2.23.0): git restore -- path/to/symlink
Switch branches (away and back)
Hard Reset: git reset --hard
Delete local repository and clone again
Troubleshooting
git config --show-scope --show-origin core.symlinks will show you the level (aka "scope") the setting is set, where the configuration file (aka "origin") that is persisting it is, and the current value of the setting. Most likely a "local" configuration is overriding the "global" or "system" setting. git config --unset core.symlinks will clear a "local" setting allowing a higher level setting to take effect.
The most recent version of Git SCM (tested on version 2.11.1) allows to enable symbolic links. But you have to clone the repository with the symbolic links again git clone -c core.symlinks=true <URL>. You need to run this command with administrator rights. It is also possible to create symbolic links on Windows with mklink.
Check out the wiki.
So as things have changed with Git since a lot of these answers were posted, here is the correct instructions to get symbolic links working correctly in Windows as of:
August 2018
1. Make sure Git is installed with symbolic link support
2. Tell Bash to create hardlinks instead of symbolic links
(git folder)/etc/bash.bashrc
Add to bottom - MSYS=winsymlinks:nativestrict
3. Set Git config to use symbolic links
git config core.symlinks true
or
git clone -c core.symlinks=true <URL>
Note: I have tried adding this to the global Git configuration and at the moment it is not working for me, so I recommend adding this to each repository...
4. pull the repository
Note: Unless you have enabled developer mode in the latest version of Windows 10, you need to run Bash as administrator to create symbolic links
5. Reset all symbolic links (optional)
If you have an existing repository, or are using submodules you may find that the symbolic links are not being created correctly so to refresh all the symbolic links in the repository you can run these commands.
find -type l -delete
git reset --hard
Note: this will reset any changes since the last commit, so make sure you have committed first
It ought to be implemented in MSysGit, but there are two downsides:
Symbolic links are only available in Windows Vista and later (it should not be an issue in 2011, and yet it is...), since older versions only support directory junctions.
(the big one) Microsoft considers symbolic links a security risk and so only administrators can create them by default. You'll need to elevate privileges of the Git process or use fstool to change this behavior on every machine you work on.
I did a quick search and there is work being actively done on this; see issue 224.
Short answer: They are now supported nicely, if you can enable developer mode.
From Symlinks in Windows 10!:
Now in Windows 10 Creators Update, a user (with admin rights) can
first enable Developer Mode, and then any user on the machine can run
the mklink command without elevating a command-line console.
What drove this change? The availability and use of symlinks is a big
deal to modern developers:
Many popular development tools like git and package managers like npm
recognize and persist symlinks when creating repos or packages,
respectively. When those repos or packages are then restored
elsewhere, the symlinks are also restored, ensuring disk space (and
the user’s time) isn’t wasted.
It is easy to overlook with all the other announcements of the "Creator's update", but if you enable Developer Mode, you can create symbolic links without elevated privileges. You might have to reinstall Git and make sure symbolic link support is enabled, as it's not by default.
I would suggest you don't use symlinks within the repository. Store the actual content inside the repository and then place symlinks out side the repository that point to the content.
So let’s say you are using a repository to compare hosting your site on a Unix-like system with hosting on Windows. Store the content in your repository, let’s say /httpRepoContent and c:\httpRepoContent with this being the folder that is synced via Git, SVN, etc.
Then, replace the content folder of you web server (/var/www and c:\program files\web server\www {names don't really matter, edit if you must}) with a symbolic link to the content in your repository. The web servers will see the content as actually in the 'right' place, but you get to use your source control.
However, if you need to use symlinks with in the repository, you will need to look into something like some sort of pre/post commit scripts. I know you can use them to do things, such as parse code files through a formatter for example, so it should be possible to convert the symlinks between platforms.
If any one knows a good place to learn how to do these scripts for the common source controls, SVN, Git, and MG, then please do add a comment.
Here is a batch script for converting symbolic link in repository, for files only, based on Josh Lee's answer. A script with some additional check for administrator rights is at https://gist.github.com/Quazistax/8daf09080bf54b4c7641.
#echo off
pushd "%~dp0"
setlocal EnableDelayedExpansion
for /f "tokens=3,*" %%e in ('git ls-files -s ^| findstr /R /C:"^120000"') do (
call :processFirstLine %%f
)
REM pause
goto :eof
:processFirstLine
#echo.
#echo FILE: %1
dir "%~f1" | find "<SYMLINK>" >NUL && (
#echo FILE already is a symlink
goto :eof
)
for /f "usebackq tokens=*" %%l in ("%~f1") do (
#echo LINK TO: %%l
del "%~f1"
if not !ERRORLEVEL! == 0 (
#echo FAILED: del
goto :eof
)
setlocal
call :expandRelative linkto "%1" "%%l"
mklink "%~f1" "!linkto!"
endlocal
if not !ERRORLEVEL! == 0 (
#echo FAILED: mklink
#echo reverting deletion...
git checkout -- "%~f1"
goto :eof
)
git update-index --assume-unchanged "%1"
if not !ERRORLEVEL! == 0 (
#echo FAILED: git update-index --assume-unchanged
goto :eof
)
#echo SUCCESS
goto :eof
)
goto :eof
:: param1 = result variable
:: param2 = reference path from which relative will be resolved
:: param3 = relative path
:expandRelative
pushd .
cd "%~dp2"
set %1=%~f3
popd
goto :eof
For those using Cygwin on Windows Vista, Windows 7, or above, the native git command can create "proper" symbolic links that are recognized by Windows apps such as Android Studio. You just need to set the CYGWIN environment variable to include winsymlinks:native or winsymlinks:nativestrict as such:
export CYGWIN="$CYGWIN winsymlinks:native"
The downside to this (and a significant one at that) is that the Cygwin shell has to be "Run as Administrator" in order for it to have the OS permissions required to create those kind of symbolic links. Once they're created, though, no special permissions are required to use them. As long they aren't changed in the repository by another developer, git thereafter runs fine with normal user permissions.
Personally, I use this only for symbolic links that are navigated by Windows applications (i.e., non-Cygwin) because of this added difficulty.
For more information on this option, see this Stack Overflow question: How to make a symbolic link with Cygwin in Windows 7
I just tried with Git 2.30.0 (released 2020-12-28).
This is not a full answer, but a few useful tidbits nonetheless. (Feel free to cannibalize for your own answer.)
Git Wiki Entry
There's a documentation link when installing Git for Windows
This link takes you here: https://github.com/git-for-windows/git/wiki/Symbolic-Links -- And this is quite a longish discussion.
FYI: There are at least three "kinds of links". And just to highlight an important aspect of this wiki entry: I didn't know this, but there are several ways all of which are "kind of" symbolic links on the surface, but on a technical level are very different:
git bash's "ln -s"
Which just copies things. Oh, boy. That was unexpected to me.
(FYI: Plain Cygwin does not do this. Mobaxterm does not do this. Instead they both create something that their stat command actually recognizes as "symbolic link".)
cmd.exe's builtin "mklink" command with the "/D" parameter
Which creates a directory symbolic link. (See the Microsoft documentation)
cmd.exe's builtin "mklink" command with the "/J" parameter.
Which creates a directory junction AKA soft link AKA reparse point. (See the Microsoft documentation.)
Release Notes Entry
Also symbolic links keep popping up in the release notes. As of 2.30.0 this here is still listed as a "Known issue":
On Windows 10 before 1703, or when Developer Mode is turned off, special permissions are required when cloning repositories with symbolic links, therefore support for symbolic links is disabled by default. Use git clone -c core.symlinks=true <URL> to enable it, see details here.
I use symbolic links all the time between my document root and Git repository directory. I like to keep them separate. On Windows I use the mklink /j option. The junction seems to let Git behave normally:
>mklink /j <location(path) of link> <source of link>
For example:
>mklink /j c:\gitRepos\Posts C:\Bitnami\wamp\apache2\htdocs\Posts
I was looking for an easy solution to deal with the Unix symbolic links on Windows. Thank you very much for the Git aliases in previous answers.
There is one little optimization that can be done to the rm-symbolic links, so that it doesn't delete the files in the destination folder in case the alias is run a second time accidentally. Please observe the new if condition in the loop to make sure the file is not already a link to a directory before the logic is run.
git config --global alias.rm-symlinks '!__git_rm_symlinks(){
for symlink in $(git ls-files -s | egrep "^120000" | cut -f2); do
*if [ -d "$symlink" ]; then
continue
fi*
git rm-symlink "$symlink"
git update-index --assume-unchanged "$symlink"
done
}; __git_rm_symlinksenter
One simple trick we use is to just call git add --all twice in a row.
For example, our Windows 7 commit script calls:
git add --all
git add --all
The first add treats the link as text and adds the folders for delete.
The second add traverses the link correctly and undoes the delete by restoring the files.
It's less elegant than some of the other proposed solutions, but it is a simple fix to some of our legacy environments that got symbolic links added.
Here's a PowerShell script to replace Unix symbolic links with Windows.
# This fixes permission denied errors you might get when
# there are Git symbolic links being used on repositories that
# you share in both POSIX (usually the host) and Windows (VM).
#
# This is not an issue if you are checking out the same
# repository separately in each platform. This is only an issue
# when it's the same working set (AKA make a change without
# committing on OS X, go to Windows VM and Git status would show
# you that change).
#
# Based on this answer on Stack Overflow: http://stackoverflow.com/a/5930443/18475
#
# No warranties. Good luck.
#
# NOTE: It must be run in elevated PowerShell
$ROOT = $PWD
$symlinks = &git ls-files -s | gawk '/120000/{print $4}'
foreach ($symlink in $symlinks) {
$content = &Get-Content $symlink
$content = $content.Replace("/", "\")
$filename = $symlink.Split("/")[-1]
cd (dirname $symlink)
rm $filename
echo Linking $content -> $filename
New-Item -ItemType SymbolicLink -Path $filename -Target $content
&git update-index --assume-unchanged $symlink
cd $ROOT
}

Does hg pull only operate on the current working directory?

I have multiple mercurial repositories and used hg clone to create backups of them on our file server. Now I want to write a batch file that updates them once a day by running hg pull -u on each subdirectory.
I want to keep this backup script as generic as possible, so it should update all backup repositories stored in my H:\BACKUPS\REPOS folder. This is my hgbackup.bat that is stored in the same folder:
for /f "delims=" %%i in ('dir /ad/b') do hg pull -u
The problem: hg pull only seems to operate on the current working directory, there seems to be no switch to specify the target repository for the pull. As I hate Windows Batch Scripting, I want to keep my .bat as simple as possible and avoid cd'ing to the different directories.
Any ideas how I can run hg pull -u on a different directory?
Use the -R-switch:
hg pull -u -R /path/to/repository
See hg -v help pull for all command line options of hg pull (the -v switch tells help to include global options).
Found this question quite a bit later due to a script I was working on for my own computer, and rather than a batch script, I did it in PowerShell (since you mentioned it's on a server, I assumed PS was available). This handles both Subversion and Mercurial repositories:
$path = "c:\users\mattgwagner\Documents\Code"
foreach($fi in get-childitem $path)
{
if(test-path $path\$fi\.svn)
{
"Updating " + $fi + " via Subversion..."
svn update $path\$fi
}
elseif(test-path $path\$fi\.hg)
{
"Updating " + $fi + " via Mercurial..."
hg pull -u -R $path\$fi
}
}

GIT clone to external drive for backup

We have GIT set up within our windows network (using msysgit & GitExtensions). We each have our own repositories and we push to a remote 'bare' repository on one of our servers. All good.
I'm trying to set up a scheduled job on the server, which will clone a repository from the C drive to an external drive (on F) - having some difficulty getting this to work. I can do this in GIT bash relatively easily, but I'm not sure how to save this into a batch file that I can then scehdule.
What I have so far:
rmdir F:\GitClone /s /q
mkdir F:\GitClone
mkdir F:\GitClone\Repo1
CD /D F:\GitClone\Repo1\
GIT CLONE /c/GIT/Repo1/
I've also tried the following for the last line:
GIT CLONE C:\GIT\Repo1\
But this doesn't work either... I'm a little stumped and would appreciate some help. The C drive contains our bare repositories and the F drive being our external drive that we swap out daily...
Several answers here that have been very useful, thanks. My resulting answer is probably a combination of these, so points for pointing out how to run a bash script and how to script the pull/push.
Need to bring these together to work so that it's happy when various drives are swapped in and out (i.e. clone a repository if it doesn't exist on the external drive and then only pull the differences otherwise), but that should be doable. Thanks to all.
Please note that git itself is excellent at copying only the needed changes to a cloned repository.
If you want a copy of your repo to be regularly updated, do this: You create a bare repository as a backup repository, and then repeatedly push all new changes there (no need to delete the old backup).
Ok, let's start by creating your repo
$ cd /tmp
$ mkdir myrepo && cd myrepo
$ touch hi && git add . && git commit -m "bla"
So, this is your repository. Now we create the clone:
$ cd /tmp
$ mkdir backup && cd backup
$ git --bare init
Initialized empty Git repository in /tmp/backup/
Now, let's set up your repo for regular backups …
$ cd /tmp/myrepo
$ git remote add backup /tmp/backup
$ git config remote.backup.mirror true
Then copy everything to the backup:
$ git push backup
Counting objects: 3, done.
Writing objects: 100% (3/3), 206 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /tmp/backup
* °new branch§ master -> master
And see if it worked:
$ cd /tmp/backup
$ git log
commit d027b125166ff3a5be2d7f7416893a012f218f82
Author: Niko Schwarz <niko.schwarzàgmail.com>
Date: Fri Dec 11 12:24:03 2009 +0100
hi
Tada, you're set. Therefore, all your script needs to do is to issue git push backup. There's exactly no need to repeatedly throw away the old backup.
The alternative is you can have rsync do it all for you:
rsync -av rsync://rsync.samba.org/ftp/unpacked/rsync /dest/dir/
User Offby adds: Since version 1.5.4, "git remote add" takes a "--mirror" option, which saves you both from having to "git config remote.origin.mirror true", and from having to pass --mirror to "git push".
Because git command is little bit weird you have to use call to execute any git commands from a batch file:
rmdir F:\GitClone /s /q
mkdir F:\GitClone
CD /D F:\GitClone\
call GIT CLONE c/GIT/Repo1/
You could always just schedule bash.exe mybashbackupscript.sh
Anyway, in windows terms:
rmdir F:\GitClone /s /q
mkdir -p F:\GitClone\Repo1
copy c:\GIT\Repo1\.git F:\GitClone\Repo1\.git
git clone doesn't really do anything fancier than that.
edit: as someone else pointed out, it's probably best to just make a new backup repo and just pull/fetch into that. You'll avoid any accesing issues with the .git :)
Sorry, I can't comment posts, but I was thinking too about copying the .git, but what happens if the .git is copyed during a pull ?
Anyway, why copying the whole stuff as you could fetch the deltas from time to time ?
Initialization of backup : (in F:\GitClone\Repo1 empty)
git init
git add remote origin /c/GIT/Repo1
Then your "delta backup script" would just do :
cd /f/GitClone/Repo1
git fetch origin
Why do you delete the clone all the time? What is the point of cloning an existing repository when you just want to copy the file?
Just create the repo on the external drive (using git clone once), and then run git pull on it regularly.
Problem solved:
h: (flash drive)
cd \
mkdir YourProject
cd YourProject
git init
git remote add origin git#github.com:user/YourProject.git
git remote add local C:\w\YourProject
git pull origin master
git push local master

Resources